2016-05-17 22:36:22,686 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(197)) - found 1 test platforms from classpath
2016-05-17 22:36:22,718 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(246)) - adding test: cascading.detail.EachPipeAssemblyPlatformTest, with platform: hadoop2-tez
2016-05-17 22:36:23,041 INFO tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-05-17 22:36:23,358 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-05-17 22:36:23,629 INFO platform.TestPlatform (TestPlatform.java:getGlobalProperties(75)) - platform property overrides:
2016-05-17 22:36:23,783 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:23,785 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:23,877 INFO property.AppProps (AppProps.java:getAppID(177)) - using app.id: DF41E04028C24EB080EAC6A437CEAF84
2016-05-17 22:36:24,099 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.063
2016-05-17 22:36:24,118 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.083
2016-05-17 22:36:24,119 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:24,120 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:24,121 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:24,285 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:24,289 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_-1
2016-05-17 22:36:24,506 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.1.0
2016-05-17 22:36:24,508 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:24,509 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:24,510 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_-1"]
2016-05-17 22:36:24,510 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:24,511 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:24,512 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:24,512 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...blyplatform/-1_'field'_-1
2016-05-17 22:36:24,513 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:24,543 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:24,723 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:24,732 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:24,732 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:24,790 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C0B2302F12A34BD0860F2B9A9E1950BB/D79E501F980C45519A762D19C55FF334/.tez/application_1463524584732_0001 doesn't exist and is created
2016-05-17 22:36:24,871 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:24,872 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:24,873 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C0B2302F12A34BD0860F2B9A9E1950BB/D79E501F980C45519A762D19C55FF334/.tez/application_1463524584732_0001
2016-05-17 22:36:24,917 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524584732_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:24,956 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:24,972 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:24,972 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-05-17 22:36:24,972 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-05-17 22:36:24,972 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:24,996 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:25,003 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:25,006 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C0B2302F12A34BD0860F2B9A9E1950BB/D79E501F980C45519A762D19C55FF334/.tez/application_1463524584732_0001/localmode-log-dir
2016-05-17 22:36:25,007 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C0B2302F12A34BD0860F2B9A9E1950BB/D79E501F980C45519A762D19C55FF334/.tez/application_1463524584732_0001/localmode-log-dir/history.txt.appattempt_1463524584732_0001_000000, maxErrors=10
2016-05-17 22:36:25,007 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:25,010 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524584732_0001_000000, appSubmitTime=1463524584900, launchTime=1463524584916
2016-05-17 22:36:25,019 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:25,022 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:25,024 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:25,024 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:25,073 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-17 22:36:25,073 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-17 22:36:25,073 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:25,082 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:37144
2016-05-17 22:36:25,122 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:25,174 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-17 22:36:25,174 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-17 22:36:25,174 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:25,206 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:25,206 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524584732_0001_000000, startTime=1463524585205
2016-05-17 22:36:25,206 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:25,274 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:25,290 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:25,290 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/-1_'field'_-1, applicationId=application_1463524584732_0001, dagName=[C0B2302F12A34BD0860F2B9A9E1950BB/D79E501F980C45519A762D19C55FF334] each/(1/1) ...blyplatform/-1_'field'_-1
2016-05-17 22:36:25,519 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [C0B2302F12A34BD0860F2B9A9E1950BB/D79E501F980C45519A762D19C55FF334] each/(1/1) ...blyplatform/-1_'field'_-1
2016-05-17 22:36:25,611 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524584732_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C0B2302F12A34BD0860F2B9A9E1950BB/D79E501F980C45519A762D19C55FF334/.tez/application_1463524584732_0001/localmode-log-dir/dag_1463524584732_0001_1.dot
2016-05-17 22:36:25,614 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C0B2302F12A34BD0860F2B9A9E1950BB/D79E501F980C45519A762D19C55FF334/.tez/application_1463524584732_0001/localmode-log-dir/dag_1463524584732_0001_1-tez-dag.pb.txt
2016-05-17 22:36:25,695 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524584732_0001_1
2016-05-17 22:36:25,695 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:25,696 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [C0B2302F12A34BD0860F2B9A9E1950BB/D79E501F980C45519A762D19C55FF334] each/(1/1) ...blyplatform/-1_'field'_-1
2016-05-17 22:36:25 Running Dag: dag_1463524584732_0001_1
2016-05-17 22:36:25,730 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:25,730 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524584732_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524584732_0001_1, submitTime=1463524585519
2016-05-17 22:36:25,775 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DC75ABF18B3E489B9870EE090C0D1D43
2016-05-17 22:36:25,778 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:25,782 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:25,782 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524584732_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524584732_0001_1, initTime=1463524585732
2016-05-17 22:36:25,783 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524584732_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:25,786 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524584732_0001_1][Event:DAG_STARTED]: dagID=dag_1463524584732_0001_1, startTime=1463524585785
2016-05-17 22:36:25,786 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:25,788 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524584732_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:25,788 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: DC75ABF18B3E489B9870EE090C0D1D43 : {745A1E8857224A7C853B27F94505CF52={InputName=745A1E8857224A7C853B27F94505CF52}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:25,789 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 745A1E8857224A7C853B27F94505CF52, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:25,789 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43]
2016-05-17 22:36:25,788 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:25,797 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43]
2016-05-17 22:36:25,798 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43]
2016-05-17 22:36:25,800 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43]. Starting root input initializers: 1
2016-05-17 22:36:25,801 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/-1_'field'_-1, applicationId=application_1463524584732_0001, dagName=[C0B2302F12A34BD0860F2B9A9E1950BB/D79E501F980C45519A762D19C55FF334] each/(1/1) ...blyplatform/-1_'field'_-1
2016-05-17 22:36:25,816 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 745A1E8857224A7C853B27F94505CF52 on vertex vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43]
2016-05-17 22:36:25,824 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:25,840 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:25,842 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 745A1E8857224A7C853B27F94505CF52 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:25,842 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524584732_0001, with dag id: dag_1463524584732_0001_1
2016-05-17 22:36:25,843 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:25,844 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:25,905 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:25,908 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 745A1E8857224A7C853B27F94505CF52 on vertex vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43]
2016-05-17 22:36:25,910 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:25,910 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {745A1E8857224A7C853B27F94505CF52=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:25,930 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43] parallelism set to 1
2016-05-17 22:36:25,930 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:25,941 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524584732_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524584732_0001_1_00, eventCount=1
2016-05-17 22:36:25,944 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43]
2016-05-17 22:36:25,945 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=1B5FEC3E1D4643D1BF8ADC9822AA06FE, vertexId=vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:25,948 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=1B5FEC3E1D4643D1BF8ADC9822AA06FE, vertexId=vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43]
2016-05-17 22:36:25,966 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=1B5FEC3E1D4643D1BF8ADC9822AA06FE, vertexName=DC75ABF18B3E489B9870EE090C0D1D43, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:25,977 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=1B5FEC3E1D4643D1BF8ADC9822AA06FE, vertexId=vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43]
2016-05-17 22:36:25,982 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:25,983 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524584732_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DC75ABF18B3E489B9870EE090C0D1D43, vertexId=vertex_1463524584732_0001_1_00, initRequestedTime=1463524585788, initedTime=1463524585980, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:25,983 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43] with distanceFromRoot: 0
2016-05-17 22:36:25,983 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:25,986 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in DC75ABF18B3E489B9870EE090C0D1D43
2016-05-17 22:36:25,986 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43]
2016-05-17 22:36:25,988 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:25,989 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524584732_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524584732_0001_1_00, startRequestedTime=1463524585985, startedTime=1463524585985
2016-05-17 22:36:25,990 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:26,027 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:26,028 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524584732_0001_1][Event:TASK_STARTED]: vertexName=DC75ABF18B3E489B9870EE090C0D1D43, taskId=task_1463524584732_0001_1_00_000000, scheduledTime=1463524586022, launchTime=1463524586022
2016-05-17 22:36:26,029 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524584732_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:26,032 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43]
2016-05-17 22:36:26,036 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C0B2302F12A34BD0860F2B9A9E1950BB/D79E501F980C45519A762D19C55FF334] each/(1/1) ...blyplatform/-1_'field'_-1, VertexName: DC75ABF18B3E489B9870EE090C0D1D43, VertexParallelism: 1, TaskAttemptID:attempt_1463524584732_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=745A1E8857224A7C853B27F94505CF52, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=1B5FEC3E1D4643D1BF8ADC9822AA06FE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:26,039 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524584732_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:26,039 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:26,078 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:26,084 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:26,085 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:26,089 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524584732_0001_00_000001
2016-05-17 22:36:26,089 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524584732_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:26,099 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524584732_0001_00_000001
2016-05-17 22:36:26,099 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:26,103 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524584732_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:26,104 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:26,104 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524584732_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524584732_0001_00_000001, launchTime=1463524586103
2016-05-17 22:36:26,108 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524584732_0001_00_000001
2016-05-17 22:36:26,111 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524584732_0001_1_00_000000_0] to container: [container_1463524584732_0001_00_000001]
2016-05-17 22:36:26,112 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524584732_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:26,113 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524584732_0001_00_000001 given task: attempt_1463524584732_0001_1_00_000000_0
2016-05-17 22:36:26,113 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524584732_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524584732_0001_1_00_000000_0
2016-05-17 22:36:26 Starting to run new task attempt: attempt_1463524584732_0001_1_00_000000_0
2016-05-17 22:36:26,114 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524584732_0001_1_00_000000_0
2016-05-17 22:36:26,114 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:26,115 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524584732_0001_1_00_000000_0] started. Is using containerId: [container_1463524584732_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:26,116 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:26,116 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:26,116 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:26,117 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:26,117 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524584732_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DC75ABF18B3E489B9870EE090C0D1D43, taskAttemptId=attempt_1463524584732_0001_1_00_000000_0, startTime=1463524586113, containerId=container_1463524584732_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524584732_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:26,118 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524584732_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:26,119 INFO counters.Limits (Limits.java:ensureInitialized(59)) - Counter limits initialized with parameters: GROUP_NAME_MAX=128, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-05-17 22:36:26,121 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:26,125 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524584732_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:26,142 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:26,142 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C0B2302F12A34BD0860F2B9A9E1950BB/D79E501F980C45519A762D19C55FF334] each/(1/1) ...blyplatform/-1_'field'_-1, VertexName: DC75ABF18B3E489B9870EE090C0D1D43, VertexParallelism: 1, TaskAttemptID:attempt_1463524584732_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=745A1E8857224A7C853B27F94505CF52, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=1B5FEC3E1D4643D1BF8ADC9822AA06FE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:26,145 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:26,148 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524584732_0001_1_00_000000_0
2016-05-17 22:36:26,149 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:26,149 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:26,155 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524584732_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:26,157 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=745A1E8857224A7C853B27F94505CF52, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:26,159 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:26,159 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=1B5FEC3E1D4643D1BF8ADC9822AA06FE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:26,160 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:26,161 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:26,161 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 1B5FEC3E1D4643D1BF8ADC9822AA06FE
2016-05-17 22:36:26,161 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:26,161 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 745A1E8857224A7C853B27F94505CF52
2016-05-17 22:36:26,166 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 1B5FEC3E1D4643D1BF8ADC9822AA06FE
2016-05-17 22:36:26,166 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 745A1E8857224A7C853B27F94505CF52
2016-05-17 22:36:26,191 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:26,217 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 745A1E8857224A7C853B27F94505CF52
2016-05-17 22:36:26,217 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 745A1E8857224A7C853B27F94505CF52
2016-05-17 22:36:26,220 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:26,239 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:26,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 1B5FEC3E1D4643D1BF8ADC9822AA06FE
2016-05-17 22:36:26,249 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DC75ABF18B3E489B9870EE090C0D1D43, ordinal: 0
2016-05-17 22:36:26,250 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DC75ABF18B3E489B9870EE090C0D1D43, mem on start (mb), free: 261, total: 281, max: 672
2016-05-17 22:36:26,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:26,251 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:26,251 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:26,252 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:26,253 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:26,259 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:26,260 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:26,260 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:26,261 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:26,262 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:26,262 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:26,263 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 1B5FEC3E1D4643D1BF8ADC9822AA06FE, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:26,263 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 745A1E8857224A7C853B27F94505CF52, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:26,263 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:26,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 745A1E8857224A7C853B27F94505CF52 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:26,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 745A1E8857224A7C853B27F94505CF52
2016-05-17 22:36:26,266 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:26,266 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 745A1E8857224A7C853B27F94505CF52
2016-05-17 22:36:26,266 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:26,267 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:26,267 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:26,269 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524584732_0001_1_00_000000_0
2016-05-17 22:36:26,270 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:26,278 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@38bae4a
2016-05-17 22:36:26,289 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@38bae4a
2016-05-17 22:36:26,290 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:26,290 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:26,308 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:26,333 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 745A1E8857224A7C853B27F94505CF52
2016-05-17 22:36:26,333 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_-1"], id: 1B5FEC3E1D4643D1BF8ADC9822AA06FE
2016-05-17 22:36:26,334 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_-1"]
2016-05-17 22:36:26,337 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:26,337 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:26,339 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DC75ABF18B3E489B9870EE090C0D1D43, all 1 inputs ready in: 00:00:00.002
2016-05-17 22:36:26,350 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:26,351 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:26,352 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524584732_0001_1_00_000000_0
2016-05-17 22:36:26,352 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524584732_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:26,354 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524584732_0001_1_00_000000_0
2016-05-17 22:36:26,354 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DC75ABF18B3E489B9870EE090C0D1D43, mem on close (mb), free: 253, total: 281, max: 672
2016-05-17 22:36:26 Completed running task attempt: attempt_1463524584732_0001_1_00_000000_0
2016-05-17 22:36:26,361 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524584732_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:26,363 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:26,369 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=24, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524586339, Process_Duration=15, Process_End_Time=1463524586354, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:26,370 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524584732_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DC75ABF18B3E489B9870EE090C0D1D43, taskAttemptId=attempt_1463524584732_0001_1_00_000000_0, startTime=1463524586113, finishTime=1463524586362, timeTaken=249, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=24, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524586339, Process_Duration=15, Process_End_Time=1463524586354, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:26,370 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:26,371 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:26,371 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524584732_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:26,372 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:26,371 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524584732_0001_00_000001
2016-05-17 22:36:26,375 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524584732_0001_1_00_000000_0
2016-05-17 22:36:26,378 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:26,382 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524584732_0001_1][Event:TASK_FINISHED]: vertexName=DC75ABF18B3E489B9870EE090C0D1D43, taskId=task_1463524584732_0001_1_00_000000, startTime=1463524586113, finishTime=1463524586377, timeTaken=264, status=SUCCEEDED, successfulAttemptID=attempt_1463524584732_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=24, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524586339, Process_Duration=15, Process_End_Time=1463524586354, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:26,387 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524584732_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:26,388 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524584732_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:26,390 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:26,390 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524584732_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524584732_0001_00_000001, stoppedTime=1463524586390, exitStatus=0
2016-05-17 22:36:26,391 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524584732_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:26,391 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43] : 1
2016-05-17 22:36:26,392 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43]
2016-05-17 22:36:26,397 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:26,401 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524584732_0001_1][Event:VERTEX_FINISHED]: vertexName=DC75ABF18B3E489B9870EE090C0D1D43, vertexId=vertex_1463524584732_0001_1_00, initRequestedTime=1463524585788, initedTime=1463524585980, startRequestedTime=1463524585985, startedTime=1463524585985, finishTime=1463524586392, timeTaken=407, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=24, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524586339, Process_Duration=15, Process_End_Time=1463524586354, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524586113, firstTasksToStart=[ task_1463524584732_0001_1_00_000000 ], lastTaskFinishTime=1463524586362, lastTasksToFinish=[ task_1463524584732_0001_1_00_000000 ], minTaskDuration=249, maxTaskDuration=249, avgTaskDuration=249.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524584732_0001_1_00_000000 ], longestDurationTasks=[ task_1463524584732_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:26,402 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:26,403 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524584732_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:36:26,404 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524584732_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:26,404 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524584732_0001_00_000001: 31 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:26,404 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524584732_0001_00_000001, Exiting
2016-05-17 22:36:26,404 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524584732_0001_00_000001
2016-05-17 22:36:26,404 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524584732_0001_00_000001
2016-05-17 22:36:26,408 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:26,408 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524584732_0001_1
2016-05-17 22:36:26,408 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524584732_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:26,409 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524584732_0001_00_000001 completed successfully
2016-05-17 22:36:26,410 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524584732_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524584732_0001_1
2016-05-17 22:36:26,411 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 1B5FEC3E1D4643D1BF8ADC9822AA06FE for vertex: vertex_1463524584732_0001_1_00 [DC75ABF18B3E489B9870EE090C0D1D43]
2016-05-17 22:36:26,429 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524584732_0001_1, queueSize=0
2016-05-17 22:36:26,430 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:26,432 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524584732_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524584732_0001_1, startTime=1463524585785, finishTime=1463524586424, timeTaken=639, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=24, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524586339, Process_Duration=15, Process_End_Time=1463524586354, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:26,433 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524584732_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:26,434 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524584732_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:26,436 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524584732_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:36:26,436 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524584732_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:36:26 Completed Dag: dag_1463524584732_0001_1
2016-05-17 22:36:26,437 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524584732_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:26,438 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524584732_0001_1_post
2016-05-17 22:36:26,438 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:26,438 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:26,449 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/-1_'field'_-1, applicationId=application_1463524584732_0001
2016-05-17 22:36:26,453 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:26,453 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:26,454 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:26,454 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:26,454 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:26,456 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:26,459 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:26,459 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:26,459 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:26,460 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:26,460 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:26,461 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:26,461 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:26,467 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C0B2302F12A34BD0860F2B9A9E1950BB/D79E501F980C45519A762D19C55FF334/.tez/application_1463524584732_0001
2016-05-17 22:36:26,468 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_-1/_temporary
2016-05-17 22:36:26,469 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:01.959
2016-05-17 22:36:26,595 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_'field'_-1
2016-05-17 22:36:26,627 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:26,627 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:26,691 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-05-17 22:36:26,707 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.042
2016-05-17 22:36:26,708 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:26,708 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:26,709 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:26,727 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:26,728 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_'field'
2016-05-17 22:36:26,760 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:26,760 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:26,761 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_'field'"]
2016-05-17 22:36:26,761 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:26,762 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:26,762 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:26,764 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...atform/-1_'field'_'field'
2016-05-17 22:36:26,764 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:26,781 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:26,787 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:26,787 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:26,787 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:26,798 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/75BED45D765B4012BD4C00EB903B10C2/BE47D3B8CDA64164ABC01ADA077C6766/.tez/application_1463524586787_0001 doesn't exist and is created
2016-05-17 22:36:26,827 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:26,827 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:26,828 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/75BED45D765B4012BD4C00EB903B10C2/BE47D3B8CDA64164ABC01ADA077C6766/.tez/application_1463524586787_0001
2016-05-17 22:36:26,833 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524586787_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:26,851 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:26,853 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:26,856 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:26,857 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:26,857 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/75BED45D765B4012BD4C00EB903B10C2/BE47D3B8CDA64164ABC01ADA077C6766/.tez/application_1463524586787_0001/localmode-log-dir
2016-05-17 22:36:26,858 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/75BED45D765B4012BD4C00EB903B10C2/BE47D3B8CDA64164ABC01ADA077C6766/.tez/application_1463524586787_0001/localmode-log-dir/history.txt.appattempt_1463524586787_0001_000000, maxErrors=10
2016-05-17 22:36:26,859 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:26,860 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524586787_0001_000000, appSubmitTime=1463524586832, launchTime=1463524586833
2016-05-17 22:36:26,870 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:26,875 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:26,877 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:26,878 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:26,889 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:56167
2016-05-17 22:36:26,893 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:26,896 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:26,896 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524586787_0001_000000, startTime=1463524586896
2016-05-17 22:36:26,896 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:26,927 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:26,928 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:26,928 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/-1_'field'_'field', applicationId=application_1463524586787_0001, dagName=[75BED45D765B4012BD4C00EB903B10C2/BE47D3B8CDA64164ABC01ADA077C6766] each/(1/1) ...atform/-1_'field'_'field'
2016-05-17 22:36:26,957 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [75BED45D765B4012BD4C00EB903B10C2/BE47D3B8CDA64164ABC01ADA077C6766] each/(1/1) ...atform/-1_'field'_'field'
2016-05-17 22:36:26,961 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524586787_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/75BED45D765B4012BD4C00EB903B10C2/BE47D3B8CDA64164ABC01ADA077C6766/.tez/application_1463524586787_0001/localmode-log-dir/dag_1463524586787_0001_1.dot
2016-05-17 22:36:26,962 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/75BED45D765B4012BD4C00EB903B10C2/BE47D3B8CDA64164ABC01ADA077C6766/.tez/application_1463524586787_0001/localmode-log-dir/dag_1463524586787_0001_1-tez-dag.pb.txt
2016-05-17 22:36:26,993 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524586787_0001_1
2016-05-17 22:36:26,993 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:26,993 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [75BED45D765B4012BD4C00EB903B10C2/BE47D3B8CDA64164ABC01ADA077C6766] each/(1/1) ...atform/-1_'field'_'field'
2016-05-17 22:36:26 Running Dag: dag_1463524586787_0001_1
2016-05-17 22:36:27,013 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:27,013 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524586787_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524586787_0001_1, submitTime=1463524586957
2016-05-17 22:36:27,015 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A24DBF751A15432F9AE23FA2EA1DBB1B
2016-05-17 22:36:27,015 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:27,015 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524586787_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524586787_0001_1, initTime=1463524587014
2016-05-17 22:36:27,016 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:27,016 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524586787_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:27,017 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524586787_0001_1][Event:DAG_STARTED]: dagID=dag_1463524586787_0001_1, startTime=1463524587017
2016-05-17 22:36:27,017 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:27,017 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524586787_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:27,017 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A24DBF751A15432F9AE23FA2EA1DBB1B : {9EE071074406497F87A40CB586F2C9BB={InputName=9EE071074406497F87A40CB586F2C9BB}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:27,018 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 9EE071074406497F87A40CB586F2C9BB, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:27,018 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B]
2016-05-17 22:36:27,018 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B]
2016-05-17 22:36:27,018 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B]
2016-05-17 22:36:27,018 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B]. Starting root input initializers: 1
2016-05-17 22:36:27,018 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:27,019 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:27,021 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/-1_'field'_'field', applicationId=application_1463524586787_0001, dagName=[75BED45D765B4012BD4C00EB903B10C2/BE47D3B8CDA64164ABC01ADA077C6766] each/(1/1) ...atform/-1_'field'_'field'
2016-05-17 22:36:27,022 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 9EE071074406497F87A40CB586F2C9BB on vertex vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B]
2016-05-17 22:36:27,030 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:27,031 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 9EE071074406497F87A40CB586F2C9BB asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:27,032 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:27,033 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:27,036 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:27,037 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 9EE071074406497F87A40CB586F2C9BB on vertex vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B]
2016-05-17 22:36:27,037 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:27,038 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {9EE071074406497F87A40CB586F2C9BB=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:27,039 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B] parallelism set to 1
2016-05-17 22:36:27,040 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:27,040 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524586787_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524586787_0001_1_00, eventCount=1
2016-05-17 22:36:27,040 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B]
2016-05-17 22:36:27,042 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=6814EC4754AA4739A444299ACADCD317, vertexId=vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:27,042 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=6814EC4754AA4739A444299ACADCD317, vertexId=vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B]
2016-05-17 22:36:27,047 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=6814EC4754AA4739A444299ACADCD317, vertexName=A24DBF751A15432F9AE23FA2EA1DBB1B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:27,047 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=6814EC4754AA4739A444299ACADCD317, vertexId=vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B]
2016-05-17 22:36:27,050 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:27,050 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524586787_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A24DBF751A15432F9AE23FA2EA1DBB1B, vertexId=vertex_1463524586787_0001_1_00, initRequestedTime=1463524587017, initedTime=1463524587050, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:27,050 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B] with distanceFromRoot: 0
2016-05-17 22:36:27,050 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:27,050 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A24DBF751A15432F9AE23FA2EA1DBB1B
2016-05-17 22:36:27,052 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B]
2016-05-17 22:36:27,053 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:27,053 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524586787_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524586787_0001_1_00, startRequestedTime=1463524587050, startedTime=1463524587050
2016-05-17 22:36:27,053 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:27,054 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:27,055 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524586787_0001_1][Event:TASK_STARTED]: vertexName=A24DBF751A15432F9AE23FA2EA1DBB1B, taskId=task_1463524586787_0001_1_00_000000, scheduledTime=1463524587054, launchTime=1463524587054
2016-05-17 22:36:27,055 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524586787_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:27,055 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B]
2016-05-17 22:36:27,056 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [75BED45D765B4012BD4C00EB903B10C2/BE47D3B8CDA64164ABC01ADA077C6766] each/(1/1) ...atform/-1_'field'_'field', VertexName: A24DBF751A15432F9AE23FA2EA1DBB1B, VertexParallelism: 1, TaskAttemptID:attempt_1463524586787_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9EE071074406497F87A40CB586F2C9BB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=6814EC4754AA4739A444299ACADCD317, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:27,057 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524586787_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:27,057 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:27,058 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:27,059 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:27,060 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:27,060 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524586787_0001_00_000001
2016-05-17 22:36:27,061 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524586787_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:27,068 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524586787_0001_00_000001
2016-05-17 22:36:27,068 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:27,069 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524586787_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:27,070 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:27,071 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524586787_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524586787_0001_00_000001, launchTime=1463524587069
2016-05-17 22:36:27,071 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524586787_0001, with dag id: dag_1463524586787_0001_1
2016-05-17 22:36:27,072 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524586787_0001_00_000001
2016-05-17 22:36:27,073 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524586787_0001_1_00_000000_0] to container: [container_1463524586787_0001_00_000001]
2016-05-17 22:36:27,074 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524586787_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:27,075 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524586787_0001_00_000001 given task: attempt_1463524586787_0001_1_00_000000_0
2016-05-17 22:36:27,075 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524586787_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524586787_0001_1_00_000000_0
2016-05-17 22:36:27,075 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524586787_0001_1_00_000000_0] started. Is using containerId: [container_1463524586787_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:27,075 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524586787_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A24DBF751A15432F9AE23FA2EA1DBB1B, taskAttemptId=attempt_1463524586787_0001_1_00_000000_0, startTime=1463524587075, containerId=container_1463524586787_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524586787_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:27,075 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524586787_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:27,076 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524586787_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:27,076 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:27 Starting to run new task attempt: attempt_1463524586787_0001_1_00_000000_0
2016-05-17 22:36:27,076 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524586787_0001_1_00_000000_0
2016-05-17 22:36:27,076 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:27,076 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:27,077 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:27,077 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:27,077 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:27,080 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:27,080 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [75BED45D765B4012BD4C00EB903B10C2/BE47D3B8CDA64164ABC01ADA077C6766] each/(1/1) ...atform/-1_'field'_'field', VertexName: A24DBF751A15432F9AE23FA2EA1DBB1B, VertexParallelism: 1, TaskAttemptID:attempt_1463524586787_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9EE071074406497F87A40CB586F2C9BB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=6814EC4754AA4739A444299ACADCD317, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:27,081 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:27,082 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524586787_0001_1_00_000000_0
2016-05-17 22:36:27,083 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524586787_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:27,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:27,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:27,085 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:27,085 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=6814EC4754AA4739A444299ACADCD317, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:27,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=9EE071074406497F87A40CB586F2C9BB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:27,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:27,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 6814EC4754AA4739A444299ACADCD317
2016-05-17 22:36:27,092 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:27,092 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 6814EC4754AA4739A444299ACADCD317
2016-05-17 22:36:27,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:27,096 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 9EE071074406497F87A40CB586F2C9BB
2016-05-17 22:36:27,097 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 9EE071074406497F87A40CB586F2C9BB
2016-05-17 22:36:27,094 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:27,112 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A24DBF751A15432F9AE23FA2EA1DBB1B, ordinal: 0
2016-05-17 22:36:27,112 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A24DBF751A15432F9AE23FA2EA1DBB1B, mem on start (mb), free: 169, total: 281, max: 672
2016-05-17 22:36:27,112 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:27,112 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:27,113 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:27,114 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 9EE071074406497F87A40CB586F2C9BB
2016-05-17 22:36:27,115 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 9EE071074406497F87A40CB586F2C9BB
2016-05-17 22:36:27,115 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:27,125 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:27,125 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 6814EC4754AA4739A444299ACADCD317
2016-05-17 22:36:27,126 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:27,126 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:27,126 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:27,126 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:27,126 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:27,126 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:27,127 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:27,127 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:27,127 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 6814EC4754AA4739A444299ACADCD317, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:27,127 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 9EE071074406497F87A40CB586F2C9BB, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:27,127 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:27,128 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 9EE071074406497F87A40CB586F2C9BB being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:27,128 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 9EE071074406497F87A40CB586F2C9BB
2016-05-17 22:36:27,128 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:27,128 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 9EE071074406497F87A40CB586F2C9BB
2016-05-17 22:36:27,128 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:27,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:27,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:27,130 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524586787_0001_1_00_000000_0
2016-05-17 22:36:27,134 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:27,135 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@a90eb7d
2016-05-17 22:36:27,138 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@a90eb7d
2016-05-17 22:36:27,139 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:27,139 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:27,145 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:27,146 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 9EE071074406497F87A40CB586F2C9BB
2016-05-17 22:36:27,146 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_'field'"], id: 6814EC4754AA4739A444299ACADCD317
2016-05-17 22:36:27,146 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_'field'"]
2016-05-17 22:36:27,147 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:27,147 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:27,147 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A24DBF751A15432F9AE23FA2EA1DBB1B, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:27,150 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:27,151 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:27,152 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524586787_0001_1_00_000000_0
2016-05-17 22:36:27,153 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524586787_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:27,154 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524586787_0001_1_00_000000_0
2016-05-17 22:36:27,155 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A24DBF751A15432F9AE23FA2EA1DBB1B, mem on close (mb), free: 166, total: 281, max: 672
2016-05-17 22:36:27 Completed running task attempt: attempt_1463524586787_0001_1_00_000000_0
2016-05-17 22:36:27,157 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524586787_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:27,158 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524587147, Process_Duration=7, Process_End_Time=1463524587154, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:27,159 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524586787_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A24DBF751A15432F9AE23FA2EA1DBB1B, taskAttemptId=attempt_1463524586787_0001_1_00_000000_0, startTime=1463524587075, finishTime=1463524587158, timeTaken=83, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524587147, Process_Duration=7, Process_End_Time=1463524587154, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:27,159 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:27,158 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:27,160 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524586787_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:27,159 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:27,160 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:27,160 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524586787_0001_1_00_000000_0
2016-05-17 22:36:27,161 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524586787_0001_00_000001
2016-05-17 22:36:27,162 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:27,162 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524586787_0001_1][Event:TASK_FINISHED]: vertexName=A24DBF751A15432F9AE23FA2EA1DBB1B, taskId=task_1463524586787_0001_1_00_000000, startTime=1463524587075, finishTime=1463524587161, timeTaken=86, status=SUCCEEDED, successfulAttemptID=attempt_1463524586787_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524587147, Process_Duration=7, Process_End_Time=1463524587154, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:27,163 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524586787_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:27,163 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B] : 1
2016-05-17 22:36:27,163 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B]
2016-05-17 22:36:27,165 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:27,166 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524586787_0001_1][Event:VERTEX_FINISHED]: vertexName=A24DBF751A15432F9AE23FA2EA1DBB1B, vertexId=vertex_1463524586787_0001_1_00, initRequestedTime=1463524587017, initedTime=1463524587050, startRequestedTime=1463524587050, startedTime=1463524587050, finishTime=1463524587163, timeTaken=113, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524587147, Process_Duration=7, Process_End_Time=1463524587154, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524587075, firstTasksToStart=[ task_1463524586787_0001_1_00_000000 ], lastTaskFinishTime=1463524587158, lastTasksToFinish=[ task_1463524586787_0001_1_00_000000 ], minTaskDuration=83, maxTaskDuration=83, avgTaskDuration=83.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524586787_0001_1_00_000000 ], longestDurationTasks=[ task_1463524586787_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:27,166 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:27,167 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524586787_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:27,167 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:27,167 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524586787_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524586787_0001_00_000001, stoppedTime=1463524587167, exitStatus=0
2016-05-17 22:36:27,169 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524586787_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:27,169 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:27,169 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524586787_0001_1
2016-05-17 22:36:27,169 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524586787_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524586787_0001_1
2016-05-17 22:36:27,170 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 6814EC4754AA4739A444299ACADCD317 for vertex: vertex_1463524586787_0001_1_00 [A24DBF751A15432F9AE23FA2EA1DBB1B]
2016-05-17 22:36:27,172 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524586787_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:27,173 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524586787_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:27,173 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524586787_0001_00_000001, Exiting
2016-05-17 22:36:27,173 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524586787_0001_00_000001
2016-05-17 22:36:27,173 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524586787_0001_00_000001
2016-05-17 22:36:27,176 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524586787_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:27,176 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524586787_0001_00_000001 completed successfully
2016-05-17 22:36:27,184 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524586787_0001_1, queueSize=0
2016-05-17 22:36:27,184 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524586787_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524586787_0001_1, startTime=1463524587017, finishTime=1463524587183, timeTaken=166, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524587147, Process_Duration=7, Process_End_Time=1463524587154, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:27,184 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524586787_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:27,185 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:27,185 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524586787_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:27,185 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524586787_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:27,186 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524586787_0001_00_000001
2016-05-17 22:36:27 Completed Dag: dag_1463524586787_0001_1
2016-05-17 22:36:27,186 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524586787_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:27,186 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524586787_0001_1_post
2016-05-17 22:36:27,186 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:27,186 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:27,195 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/-1_'field'_'field', applicationId=application_1463524586787_0001
2016-05-17 22:36:27,198 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:27,198 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:27,198 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:27,198 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:27,199 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:27,199 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:27,200 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:27,200 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:27,200 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:27,201 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:27,201 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:27,202 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:27,202 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:27,207 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/75BED45D765B4012BD4C00EB903B10C2/BE47D3B8CDA64164ABC01ADA077C6766/.tez/application_1463524586787_0001
2016-05-17 22:36:27,208 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_'field'/_temporary
2016-05-17 22:36:27,208 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.447
2016-05-17 22:36:27,343 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_'field'_'field'
2016-05-17 22:36:27,377 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:27,378 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:27,443 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-05-17 22:36:27,456 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.037
2016-05-17 22:36:27,457 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:27,458 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:27,458 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:27,478 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:27,479 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_RESULTS
2016-05-17 22:36:27,506 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:27,507 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:27,507 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_RESULTS"]
2016-05-17 22:36:27,508 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:27,508 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:27,509 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:27,511 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...atform/-1_'field'_RESULTS
2016-05-17 22:36:27,511 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:27,530 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:27,535 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:27,536 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:27,536 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:27,546 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2C8EBB9D0C304947B5682B3A65DA9830/DA4732EFFBEF49EE8D3CE91E1632156C/.tez/application_1463524587536_0001 doesn't exist and is created
2016-05-17 22:36:27,569 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:27,570 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:27,571 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2C8EBB9D0C304947B5682B3A65DA9830/DA4732EFFBEF49EE8D3CE91E1632156C/.tez/application_1463524587536_0001
2016-05-17 22:36:27,578 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524587536_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:27,590 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:27,592 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:27,594 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:27,594 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:27,594 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2C8EBB9D0C304947B5682B3A65DA9830/DA4732EFFBEF49EE8D3CE91E1632156C/.tez/application_1463524587536_0001/localmode-log-dir
2016-05-17 22:36:27,594 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2C8EBB9D0C304947B5682B3A65DA9830/DA4732EFFBEF49EE8D3CE91E1632156C/.tez/application_1463524587536_0001/localmode-log-dir/history.txt.appattempt_1463524587536_0001_000000, maxErrors=10
2016-05-17 22:36:27,595 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:27,595 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524587536_0001_000000, appSubmitTime=1463524587577, launchTime=1463524587577
2016-05-17 22:36:27,598 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:27,603 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:27,604 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:27,605 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:27,607 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:60903
2016-05-17 22:36:27,608 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:27,611 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:27,611 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524587536_0001_000000, startTime=1463524587611
2016-05-17 22:36:27,611 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:27,670 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:27,670 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:27,670 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/-1_'field'_RESULTS, applicationId=application_1463524587536_0001, dagName=[2C8EBB9D0C304947B5682B3A65DA9830/DA4732EFFBEF49EE8D3CE91E1632156C] each/(1/1) ...atform/-1_'field'_RESULTS
2016-05-17 22:36:27,692 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [2C8EBB9D0C304947B5682B3A65DA9830/DA4732EFFBEF49EE8D3CE91E1632156C] each/(1/1) ...atform/-1_'field'_RESULTS
2016-05-17 22:36:27,693 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524587536_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2C8EBB9D0C304947B5682B3A65DA9830/DA4732EFFBEF49EE8D3CE91E1632156C/.tez/application_1463524587536_0001/localmode-log-dir/dag_1463524587536_0001_1.dot
2016-05-17 22:36:27,695 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2C8EBB9D0C304947B5682B3A65DA9830/DA4732EFFBEF49EE8D3CE91E1632156C/.tez/application_1463524587536_0001/localmode-log-dir/dag_1463524587536_0001_1-tez-dag.pb.txt
2016-05-17 22:36:27,720 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524587536_0001_1
2016-05-17 22:36:27,720 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:27,720 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [2C8EBB9D0C304947B5682B3A65DA9830/DA4732EFFBEF49EE8D3CE91E1632156C] each/(1/1) ...atform/-1_'field'_RESULTS
2016-05-17 22:36:27 Running Dag: dag_1463524587536_0001_1
2016-05-17 22:36:27,734 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:27,735 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524587536_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524587536_0001_1, submitTime=1463524587692
2016-05-17 22:36:27,735 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E0ABEF5E52634543B839DF68D44E361A
2016-05-17 22:36:27,736 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:27,736 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:27,737 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524587536_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524587536_0001_1, initTime=1463524587735
2016-05-17 22:36:27,737 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524587536_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:27,738 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524587536_0001_1][Event:DAG_STARTED]: dagID=dag_1463524587536_0001_1, startTime=1463524587738
2016-05-17 22:36:27,738 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:27,738 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:27,738 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524587536_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:27,739 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E0ABEF5E52634543B839DF68D44E361A : {91B9135F736F44B181531609015434D0={InputName=91B9135F736F44B181531609015434D0}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:27,738 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/-1_'field'_RESULTS, applicationId=application_1463524587536_0001, dagName=[2C8EBB9D0C304947B5682B3A65DA9830/DA4732EFFBEF49EE8D3CE91E1632156C] each/(1/1) ...atform/-1_'field'_RESULTS
2016-05-17 22:36:27,739 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 91B9135F736F44B181531609015434D0, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:27,739 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A]
2016-05-17 22:36:27,739 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A]
2016-05-17 22:36:27,739 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A]
2016-05-17 22:36:27,740 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A]. Starting root input initializers: 1
2016-05-17 22:36:27,740 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:27,741 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 91B9135F736F44B181531609015434D0 on vertex vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A]
2016-05-17 22:36:27,743 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:27,743 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 91B9135F736F44B181531609015434D0 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:27,744 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:27,745 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:27,747 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:27,747 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 91B9135F736F44B181531609015434D0 on vertex vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A]
2016-05-17 22:36:27,747 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:27,748 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {91B9135F736F44B181531609015434D0=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:27,749 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A] parallelism set to 1
2016-05-17 22:36:27,749 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:27,749 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524587536_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524587536_0001_1_00, eventCount=1
2016-05-17 22:36:27,749 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A]
2016-05-17 22:36:27,750 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=6BAD5D15F4D94F6799C35C831AD06B9D, vertexId=vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:27,750 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=6BAD5D15F4D94F6799C35C831AD06B9D, vertexId=vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A]
2016-05-17 22:36:27,752 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=6BAD5D15F4D94F6799C35C831AD06B9D, vertexName=E0ABEF5E52634543B839DF68D44E361A, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:27,752 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=6BAD5D15F4D94F6799C35C831AD06B9D, vertexId=vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A]
2016-05-17 22:36:27,754 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524587536_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E0ABEF5E52634543B839DF68D44E361A, vertexId=vertex_1463524587536_0001_1_00, initRequestedTime=1463524587739, initedTime=1463524587754, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:27,754 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A] with distanceFromRoot: 0
2016-05-17 22:36:27,755 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:27,755 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E0ABEF5E52634543B839DF68D44E361A
2016-05-17 22:36:27,755 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A]
2016-05-17 22:36:27,755 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524587536_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524587536_0001_1_00, startRequestedTime=1463524587755, startedTime=1463524587755
2016-05-17 22:36:27,756 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:27,756 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524587536_0001_1][Event:TASK_STARTED]: vertexName=E0ABEF5E52634543B839DF68D44E361A, taskId=task_1463524587536_0001_1_00_000000, scheduledTime=1463524587756, launchTime=1463524587756
2016-05-17 22:36:27,756 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524587536_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:27,756 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A]
2016-05-17 22:36:27,757 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2C8EBB9D0C304947B5682B3A65DA9830/DA4732EFFBEF49EE8D3CE91E1632156C] each/(1/1) ...atform/-1_'field'_RESULTS, VertexName: E0ABEF5E52634543B839DF68D44E361A, VertexParallelism: 1, TaskAttemptID:attempt_1463524587536_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=91B9135F736F44B181531609015434D0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=6BAD5D15F4D94F6799C35C831AD06B9D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:27,757 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524587536_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:27,757 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:27,758 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:27,758 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:27,758 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:27,759 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:27,763 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524587536_0001, with dag id: dag_1463524587536_0001_1
2016-05-17 22:36:27,763 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:27,763 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:27,764 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524587536_0001_00_000001
2016-05-17 22:36:27,765 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524587536_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:27,767 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524587536_0001_00_000001
2016-05-17 22:36:27,767 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:27,772 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524587536_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:27,773 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:27,776 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524587536_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524587536_0001_00_000001, launchTime=1463524587772
2016-05-17 22:36:27,777 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524587536_0001_00_000001
2016-05-17 22:36:27,779 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524587536_0001_1_00_000000_0] to container: [container_1463524587536_0001_00_000001]
2016-05-17 22:36:27,779 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524587536_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:27,779 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524587536_0001_00_000001 given task: attempt_1463524587536_0001_1_00_000000_0
2016-05-17 22:36:27,780 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524587536_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524587536_0001_1_00_000000_0
2016-05-17 22:36:27,780 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524587536_0001_1_00_000000_0] started. Is using containerId: [container_1463524587536_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:27,780 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:27 Starting to run new task attempt: attempt_1463524587536_0001_1_00_000000_0
2016-05-17 22:36:27,780 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524587536_0001_1_00_000000_0
2016-05-17 22:36:27,780 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:27,781 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:27,781 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:27,781 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:27,780 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524587536_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E0ABEF5E52634543B839DF68D44E361A, taskAttemptId=attempt_1463524587536_0001_1_00_000000_0, startTime=1463524587779, containerId=container_1463524587536_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524587536_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:27,781 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:27,781 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524587536_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:27,781 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:27,782 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524587536_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:27,782 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2C8EBB9D0C304947B5682B3A65DA9830/DA4732EFFBEF49EE8D3CE91E1632156C] each/(1/1) ...atform/-1_'field'_RESULTS, VertexName: E0ABEF5E52634543B839DF68D44E361A, VertexParallelism: 1, TaskAttemptID:attempt_1463524587536_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=91B9135F736F44B181531609015434D0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=6BAD5D15F4D94F6799C35C831AD06B9D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:27,782 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:27,783 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524587536_0001_1_00_000000_0
2016-05-17 22:36:27,787 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:27,787 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:27,787 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524587536_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:27,788 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=91B9135F736F44B181531609015434D0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:27,788 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=6BAD5D15F4D94F6799C35C831AD06B9D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:27,788 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:27,789 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:27,788 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:27,790 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 91B9135F736F44B181531609015434D0
2016-05-17 22:36:27,791 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 91B9135F736F44B181531609015434D0
2016-05-17 22:36:27,790 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 6BAD5D15F4D94F6799C35C831AD06B9D
2016-05-17 22:36:27,791 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:27,791 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 6BAD5D15F4D94F6799C35C831AD06B9D
2016-05-17 22:36:27,800 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:27,801 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:27,801 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 91B9135F736F44B181531609015434D0
2016-05-17 22:36:27,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 91B9135F736F44B181531609015434D0
2016-05-17 22:36:27,816 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:27,816 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 6BAD5D15F4D94F6799C35C831AD06B9D
2016-05-17 22:36:27,822 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E0ABEF5E52634543B839DF68D44E361A, ordinal: 0
2016-05-17 22:36:27,822 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E0ABEF5E52634543B839DF68D44E361A, mem on start (mb), free: 203, total: 281, max: 672
2016-05-17 22:36:27,822 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:27,822 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:27,822 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:27,822 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:27,822 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:27,823 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:27,824 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:27,824 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:27,824 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:27,825 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:27,825 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:27,825 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 91B9135F736F44B181531609015434D0, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:27,825 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 6BAD5D15F4D94F6799C35C831AD06B9D, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:27,826 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:27,826 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 91B9135F736F44B181531609015434D0
2016-05-17 22:36:27,826 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:27,826 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 91B9135F736F44B181531609015434D0
2016-05-17 22:36:27,827 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 91B9135F736F44B181531609015434D0 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:27,827 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:27,828 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:27,828 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:27,830 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524587536_0001_1_00_000000_0
2016-05-17 22:36:27,830 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:27,831 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@66568dc9
2016-05-17 22:36:27,834 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@66568dc9
2016-05-17 22:36:27,834 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:27,834 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:27,835 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 91B9135F736F44B181531609015434D0
2016-05-17 22:36:27,835 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_RESULTS"], id: 6BAD5D15F4D94F6799C35C831AD06B9D
2016-05-17 22:36:27,834 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:27,836 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_RESULTS"]
2016-05-17 22:36:27,837 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:27,837 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:27,837 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E0ABEF5E52634543B839DF68D44E361A, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:27,842 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:27,843 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:27,844 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524587536_0001_1_00_000000_0
2016-05-17 22:36:27,844 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524587536_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:27,846 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524587536_0001_1_00_000000_0
2016-05-17 22:36:27,846 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E0ABEF5E52634543B839DF68D44E361A, mem on close (mb), free: 201, total: 281, max: 672
2016-05-17 22:36:27 Completed running task attempt: attempt_1463524587536_0001_1_00_000000_0
2016-05-17 22:36:27,847 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524587536_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:27,849 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524587837, Process_Duration=9, Process_End_Time=1463524587846, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:27,849 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:27,850 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:27,850 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524587536_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E0ABEF5E52634543B839DF68D44E361A, taskAttemptId=attempt_1463524587536_0001_1_00_000000_0, startTime=1463524587779, finishTime=1463524587849, timeTaken=70, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524587837, Process_Duration=9, Process_End_Time=1463524587846, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:27,851 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524587536_0001_00_000001
2016-05-17 22:36:27,849 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:27,851 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524587536_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:27,852 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:27,852 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524587536_0001_1_00_000000_0
2016-05-17 22:36:27,853 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:27,854 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524587536_0001_1][Event:TASK_FINISHED]: vertexName=E0ABEF5E52634543B839DF68D44E361A, taskId=task_1463524587536_0001_1_00_000000, startTime=1463524587779, finishTime=1463524587853, timeTaken=74, status=SUCCEEDED, successfulAttemptID=attempt_1463524587536_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524587837, Process_Duration=9, Process_End_Time=1463524587846, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:27,855 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524587536_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:27,855 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A] : 1
2016-05-17 22:36:27,855 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A]
2016-05-17 22:36:27,856 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:27,857 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524587536_0001_1][Event:VERTEX_FINISHED]: vertexName=E0ABEF5E52634543B839DF68D44E361A, vertexId=vertex_1463524587536_0001_1_00, initRequestedTime=1463524587739, initedTime=1463524587754, startRequestedTime=1463524587755, startedTime=1463524587755, finishTime=1463524587855, timeTaken=100, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524587837, Process_Duration=9, Process_End_Time=1463524587846, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524587779, firstTasksToStart=[ task_1463524587536_0001_1_00_000000 ], lastTaskFinishTime=1463524587849, lastTasksToFinish=[ task_1463524587536_0001_1_00_000000 ], minTaskDuration=70, maxTaskDuration=70, avgTaskDuration=70.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524587536_0001_1_00_000000 ], longestDurationTasks=[ task_1463524587536_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:27,858 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:27,858 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524587536_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:27,858 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524587536_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524587536_0001_00_000001, stoppedTime=1463524587858, exitStatus=0
2016-05-17 22:36:27,858 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:27,859 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524587536_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:27,860 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:27,860 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524587536_0001_1
2016-05-17 22:36:27,861 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524587536_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524587536_0001_1
2016-05-17 22:36:27,861 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 6BAD5D15F4D94F6799C35C831AD06B9D for vertex: vertex_1463524587536_0001_1_00 [E0ABEF5E52634543B839DF68D44E361A]
2016-05-17 22:36:27,865 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524587536_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:27,865 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524587536_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:27,865 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524587536_0001_00_000001, Exiting
2016-05-17 22:36:27,865 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524587536_0001_00_000001
2016-05-17 22:36:27,865 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524587536_0001_00_000001
2016-05-17 22:36:27,868 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524587536_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:27,868 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524587536_0001_00_000001 completed successfully
2016-05-17 22:36:27,874 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524587536_0001_1, queueSize=0
2016-05-17 22:36:27,874 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:27,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524587536_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524587536_0001_1, startTime=1463524587738, finishTime=1463524587872, timeTaken=134, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524587837, Process_Duration=9, Process_End_Time=1463524587846, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:27,875 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524587536_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:27,875 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524587536_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:27,875 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524587536_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:27 Completed Dag: dag_1463524587536_0001_1
2016-05-17 22:36:27,876 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524587536_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:27,876 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524587536_0001_00_000001
2016-05-17 22:36:27,876 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524587536_0001_1_post
2016-05-17 22:36:27,876 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:27,877 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:27,885 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/-1_'field'_RESULTS, applicationId=application_1463524587536_0001
2016-05-17 22:36:27,889 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:27,889 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:27,889 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:27,889 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:27,889 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:27,890 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:27,892 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:27,892 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:27,892 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:27,893 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:27,893 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:27,894 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:27,896 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:27,907 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2C8EBB9D0C304947B5682B3A65DA9830/DA4732EFFBEF49EE8D3CE91E1632156C/.tez/application_1463524587536_0001
2016-05-17 22:36:27,908 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_RESULTS/_temporary
2016-05-17 22:36:27,910 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.402
2016-05-17 22:36:28,008 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_'field'_RESULTS
2016-05-17 22:36:28,038 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:28,038 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:28,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-17 22:36:28,115 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.036
2016-05-17 22:36:28,116 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:28,116 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:28,116 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:28,135 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:28,136 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_ALL
2016-05-17 22:36:28,158 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:28,158 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:28,159 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_ALL"]
2016-05-17 22:36:28,159 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:28,160 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:28,160 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:28,161 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...lyplatform/-1_'field'_ALL
2016-05-17 22:36:28,162 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:28,183 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:28,189 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:28,189 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:28,189 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:28,201 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/64EF14D202514511BA97DF54F6196EDC/DFFF8A5F7A954811B13CE34D4F75144D/.tez/application_1463524588189_0001 doesn't exist and is created
2016-05-17 22:36:28,218 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:28,218 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:28,219 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/64EF14D202514511BA97DF54F6196EDC/DFFF8A5F7A954811B13CE34D4F75144D/.tez/application_1463524588189_0001
2016-05-17 22:36:28,223 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524588189_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:28,237 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:28,239 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:28,241 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:28,241 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:28,241 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/64EF14D202514511BA97DF54F6196EDC/DFFF8A5F7A954811B13CE34D4F75144D/.tez/application_1463524588189_0001/localmode-log-dir
2016-05-17 22:36:28,242 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/64EF14D202514511BA97DF54F6196EDC/DFFF8A5F7A954811B13CE34D4F75144D/.tez/application_1463524588189_0001/localmode-log-dir/history.txt.appattempt_1463524588189_0001_000000, maxErrors=10
2016-05-17 22:36:28,242 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:28,243 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524588189_0001_000000, appSubmitTime=1463524588223, launchTime=1463524588223
2016-05-17 22:36:28,253 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:28,254 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:28,258 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:28,258 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:28,259 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:40281
2016-05-17 22:36:28,261 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:28,263 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524588189_0001_000000, startTime=1463524588263
2016-05-17 22:36:28,263 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:28,263 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:28,319 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:28,319 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:28,319 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/-1_'field'_ALL, applicationId=application_1463524588189_0001, dagName=[64EF14D202514511BA97DF54F6196EDC/DFFF8A5F7A954811B13CE34D4F75144D] each/(1/1) ...lyplatform/-1_'field'_ALL
2016-05-17 22:36:28,367 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [64EF14D202514511BA97DF54F6196EDC/DFFF8A5F7A954811B13CE34D4F75144D] each/(1/1) ...lyplatform/-1_'field'_ALL
2016-05-17 22:36:28,369 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524588189_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/64EF14D202514511BA97DF54F6196EDC/DFFF8A5F7A954811B13CE34D4F75144D/.tez/application_1463524588189_0001/localmode-log-dir/dag_1463524588189_0001_1.dot
2016-05-17 22:36:28,370 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/64EF14D202514511BA97DF54F6196EDC/DFFF8A5F7A954811B13CE34D4F75144D/.tez/application_1463524588189_0001/localmode-log-dir/dag_1463524588189_0001_1-tez-dag.pb.txt
2016-05-17 22:36:28,404 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524588189_0001_1
2016-05-17 22:36:28,404 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:28,404 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [64EF14D202514511BA97DF54F6196EDC/DFFF8A5F7A954811B13CE34D4F75144D] each/(1/1) ...lyplatform/-1_'field'_ALL
2016-05-17 22:36:28 Running Dag: dag_1463524588189_0001_1
2016-05-17 22:36:28,417 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588189_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524588189_0001_1, submitTime=1463524588368
2016-05-17 22:36:28,417 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:28,418 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BE7018F64E0C4DC489C47E03C9A84E4F
2016-05-17 22:36:28,419 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:28,419 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:28,420 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588189_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524588189_0001_1, initTime=1463524588418
2016-05-17 22:36:28,420 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524588189_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:28,421 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588189_0001_1][Event:DAG_STARTED]: dagID=dag_1463524588189_0001_1, startTime=1463524588421
2016-05-17 22:36:28,421 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:28,421 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:28,421 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524588189_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:28,421 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BE7018F64E0C4DC489C47E03C9A84E4F : {F9051DFB06C740A9AE91F888F07EF444={InputName=F9051DFB06C740A9AE91F888F07EF444}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:28,422 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: F9051DFB06C740A9AE91F888F07EF444, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:28,422 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F]
2016-05-17 22:36:28,422 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/-1_'field'_ALL, applicationId=application_1463524588189_0001, dagName=[64EF14D202514511BA97DF54F6196EDC/DFFF8A5F7A954811B13CE34D4F75144D] each/(1/1) ...lyplatform/-1_'field'_ALL
2016-05-17 22:36:28,422 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F]
2016-05-17 22:36:28,422 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F]
2016-05-17 22:36:28,423 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F]. Starting root input initializers: 1
2016-05-17 22:36:28,423 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:28,423 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: F9051DFB06C740A9AE91F888F07EF444 on vertex vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F]
2016-05-17 22:36:28,425 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:28,425 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input F9051DFB06C740A9AE91F888F07EF444 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:28,428 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:28,428 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:28,431 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:28,431 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: F9051DFB06C740A9AE91F888F07EF444 on vertex vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F]
2016-05-17 22:36:28,431 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:28,432 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {F9051DFB06C740A9AE91F888F07EF444=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:28,432 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F] parallelism set to 1
2016-05-17 22:36:28,432 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:28,432 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588189_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524588189_0001_1_00, eventCount=1
2016-05-17 22:36:28,432 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F]
2016-05-17 22:36:28,433 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=F456CE74923A47CDBB3B8F5C4D996A71, vertexId=vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:28,433 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=F456CE74923A47CDBB3B8F5C4D996A71, vertexId=vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F]
2016-05-17 22:36:28,435 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=F456CE74923A47CDBB3B8F5C4D996A71, vertexName=BE7018F64E0C4DC489C47E03C9A84E4F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:28,435 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=F456CE74923A47CDBB3B8F5C4D996A71, vertexId=vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F]
2016-05-17 22:36:28,437 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:28,437 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588189_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BE7018F64E0C4DC489C47E03C9A84E4F, vertexId=vertex_1463524588189_0001_1_00, initRequestedTime=1463524588421, initedTime=1463524588437, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:28,437 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F] with distanceFromRoot: 0
2016-05-17 22:36:28,437 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:28,438 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BE7018F64E0C4DC489C47E03C9A84E4F
2016-05-17 22:36:28,438 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F]
2016-05-17 22:36:28,438 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:28,438 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588189_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524588189_0001_1_00, startRequestedTime=1463524588438, startedTime=1463524588438
2016-05-17 22:36:28,439 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:28,439 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:28,440 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588189_0001_1][Event:TASK_STARTED]: vertexName=BE7018F64E0C4DC489C47E03C9A84E4F, taskId=task_1463524588189_0001_1_00_000000, scheduledTime=1463524588439, launchTime=1463524588439
2016-05-17 22:36:28,440 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524588189_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:28,440 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F]
2016-05-17 22:36:28,440 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [64EF14D202514511BA97DF54F6196EDC/DFFF8A5F7A954811B13CE34D4F75144D] each/(1/1) ...lyplatform/-1_'field'_ALL, VertexName: BE7018F64E0C4DC489C47E03C9A84E4F, VertexParallelism: 1, TaskAttemptID:attempt_1463524588189_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F9051DFB06C740A9AE91F888F07EF444, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F456CE74923A47CDBB3B8F5C4D996A71, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:28,441 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524588189_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:28,441 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:28,442 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:28,443 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:28,443 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:28,444 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524588189_0001_00_000001
2016-05-17 22:36:28,445 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524588189_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:28,447 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524588189_0001_00_000001
2016-05-17 22:36:28,447 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:28,448 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524588189_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:28,448 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:28,449 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524588189_0001, with dag id: dag_1463524588189_0001_1
2016-05-17 22:36:28,449 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588189_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524588189_0001_00_000001, launchTime=1463524588448
2016-05-17 22:36:28,451 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524588189_0001_00_000001
2016-05-17 22:36:28,452 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524588189_0001_1_00_000000_0] to container: [container_1463524588189_0001_00_000001]
2016-05-17 22:36:28,452 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524588189_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:28,453 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524588189_0001_00_000001 given task: attempt_1463524588189_0001_1_00_000000_0
2016-05-17 22:36:28,453 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524588189_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524588189_0001_1_00_000000_0
2016-05-17 22:36:28,454 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524588189_0001_1_00_000000_0] started. Is using containerId: [container_1463524588189_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:28 Starting to run new task attempt: attempt_1463524588189_0001_1_00_000000_0
2016-05-17 22:36:28,454 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524588189_0001_1_00_000000_0
2016-05-17 22:36:28,454 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588189_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BE7018F64E0C4DC489C47E03C9A84E4F, taskAttemptId=attempt_1463524588189_0001_1_00_000000_0, startTime=1463524588453, containerId=container_1463524588189_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524588189_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:28,454 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524588189_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:28,455 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524588189_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:28,455 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:28,455 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:28,455 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:28,455 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:28,456 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:28,456 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:28,456 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:28,456 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [64EF14D202514511BA97DF54F6196EDC/DFFF8A5F7A954811B13CE34D4F75144D] each/(1/1) ...lyplatform/-1_'field'_ALL, VertexName: BE7018F64E0C4DC489C47E03C9A84E4F, VertexParallelism: 1, TaskAttemptID:attempt_1463524588189_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F9051DFB06C740A9AE91F888F07EF444, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F456CE74923A47CDBB3B8F5C4D996A71, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:28,456 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:28,457 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524588189_0001_1_00_000000_0
2016-05-17 22:36:28,458 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:28,458 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:28,459 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524588189_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:28,459 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:28,459 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F9051DFB06C740A9AE91F888F07EF444, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:28,462 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:28,463 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F9051DFB06C740A9AE91F888F07EF444
2016-05-17 22:36:28,463 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: F9051DFB06C740A9AE91F888F07EF444
2016-05-17 22:36:28,460 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F456CE74923A47CDBB3B8F5C4D996A71, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:28,462 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:28,465 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:28,465 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F456CE74923A47CDBB3B8F5C4D996A71
2016-05-17 22:36:28,465 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:28,465 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: F456CE74923A47CDBB3B8F5C4D996A71
2016-05-17 22:36:28,469 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:28,469 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: F9051DFB06C740A9AE91F888F07EF444
2016-05-17 22:36:28,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F9051DFB06C740A9AE91F888F07EF444
2016-05-17 22:36:28,479 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:28,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F456CE74923A47CDBB3B8F5C4D996A71
2016-05-17 22:36:28,480 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BE7018F64E0C4DC489C47E03C9A84E4F, ordinal: 0
2016-05-17 22:36:28,480 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BE7018F64E0C4DC489C47E03C9A84E4F, mem on start (mb), free: 345, total: 389, max: 672
2016-05-17 22:36:28,481 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:28,481 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:28,481 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:28,482 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:28,482 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:28,482 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:28,483 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:28,483 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:28,483 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:28,483 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:28,484 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:28,484 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F9051DFB06C740A9AE91F888F07EF444, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:28,484 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F456CE74923A47CDBB3B8F5C4D996A71, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:28,484 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:28,485 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F9051DFB06C740A9AE91F888F07EF444 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:28,485 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F9051DFB06C740A9AE91F888F07EF444
2016-05-17 22:36:28,485 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:28,485 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F9051DFB06C740A9AE91F888F07EF444
2016-05-17 22:36:28,485 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:28,486 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:28,486 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:28,487 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524588189_0001_1_00_000000_0
2016-05-17 22:36:28,487 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:28,488 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5dc7b741
2016-05-17 22:36:28,489 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5dc7b741
2016-05-17 22:36:28,489 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:28,489 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:28,491 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:28,492 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: F9051DFB06C740A9AE91F888F07EF444
2016-05-17 22:36:28,492 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_ALL"], id: F456CE74923A47CDBB3B8F5C4D996A71
2016-05-17 22:36:28,492 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_ALL"]
2016-05-17 22:36:28,493 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:28,493 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:28,493 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BE7018F64E0C4DC489C47E03C9A84E4F, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:28,497 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:28,497 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:28,498 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524588189_0001_1_00_000000_0
2016-05-17 22:36:28,498 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524588189_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:28,499 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524588189_0001_1_00_000000_0
2016-05-17 22:36:28,499 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BE7018F64E0C4DC489C47E03C9A84E4F, mem on close (mb), free: 343, total: 389, max: 672
2016-05-17 22:36:28 Completed running task attempt: attempt_1463524588189_0001_1_00_000000_0
2016-05-17 22:36:28,502 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524588189_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:28,503 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=758, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=408420352, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524588494, Process_Duration=5, Process_End_Time=1463524588499, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:28,504 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:28,504 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588189_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BE7018F64E0C4DC489C47E03C9A84E4F, taskAttemptId=attempt_1463524588189_0001_1_00_000000_0, startTime=1463524588453, finishTime=1463524588503, timeTaken=50, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=758, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=408420352, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524588494, Process_Duration=5, Process_End_Time=1463524588499, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:28,505 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524588189_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:28,506 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:28,506 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524588189_0001_1_00_000000_0
2016-05-17 22:36:28,506 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:28,507 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:28,507 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524588189_0001_00_000001
2016-05-17 22:36:28,507 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:28,507 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588189_0001_1][Event:TASK_FINISHED]: vertexName=BE7018F64E0C4DC489C47E03C9A84E4F, taskId=task_1463524588189_0001_1_00_000000, startTime=1463524588453, finishTime=1463524588506, timeTaken=53, status=SUCCEEDED, successfulAttemptID=attempt_1463524588189_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=758, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=408420352, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524588494, Process_Duration=5, Process_End_Time=1463524588499, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:28,508 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524588189_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:28,508 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F] : 1
2016-05-17 22:36:28,509 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F]
2016-05-17 22:36:28,510 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:28,511 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588189_0001_1][Event:VERTEX_FINISHED]: vertexName=BE7018F64E0C4DC489C47E03C9A84E4F, vertexId=vertex_1463524588189_0001_1_00, initRequestedTime=1463524588421, initedTime=1463524588437, startRequestedTime=1463524588438, startedTime=1463524588438, finishTime=1463524588509, timeTaken=71, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=758, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=408420352, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524588494, Process_Duration=5, Process_End_Time=1463524588499, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524588453, firstTasksToStart=[ task_1463524588189_0001_1_00_000000 ], lastTaskFinishTime=1463524588503, lastTasksToFinish=[ task_1463524588189_0001_1_00_000000 ], minTaskDuration=50, maxTaskDuration=50, avgTaskDuration=50.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524588189_0001_1_00_000000 ], longestDurationTasks=[ task_1463524588189_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:28,511 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:28,512 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524588189_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:28,512 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:28,512 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588189_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524588189_0001_00_000001, stoppedTime=1463524588512, exitStatus=0
2016-05-17 22:36:28,513 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524588189_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:28,513 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:28,513 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524588189_0001_1
2016-05-17 22:36:28,514 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588189_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524588189_0001_1
2016-05-17 22:36:28,514 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: F456CE74923A47CDBB3B8F5C4D996A71 for vertex: vertex_1463524588189_0001_1_00 [BE7018F64E0C4DC489C47E03C9A84E4F]
2016-05-17 22:36:28,520 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524588189_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:28,520 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524588189_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:28,520 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524588189_0001_00_000001, Exiting
2016-05-17 22:36:28,521 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524588189_0001_00_000001
2016-05-17 22:36:28,521 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524588189_0001_00_000001
2016-05-17 22:36:28,523 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524588189_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:28,524 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524588189_0001_00_000001 completed successfully
2016-05-17 22:36:28,527 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524588189_0001_1, queueSize=0
2016-05-17 22:36:28,527 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:28,529 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588189_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524588189_0001_1, startTime=1463524588421, finishTime=1463524588526, timeTaken=105, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=758, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=408420352, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524588494, Process_Duration=5, Process_End_Time=1463524588499, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:28,529 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524588189_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:28,529 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524588189_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:28,530 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524588189_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:28,530 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524588189_0001_00_000001
2016-05-17 22:36:28 Completed Dag: dag_1463524588189_0001_1
2016-05-17 22:36:28,533 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524588189_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:28,534 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524588189_0001_1_post
2016-05-17 22:36:28,534 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:28,534 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:28,537 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/-1_'field'_ALL, applicationId=application_1463524588189_0001
2016-05-17 22:36:28,540 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:28,540 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:28,540 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:28,540 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:28,541 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:28,541 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:28,541 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:28,542 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:28,542 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:28,542 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:28,543 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:28,543 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:28,544 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:28,548 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/64EF14D202514511BA97DF54F6196EDC/DFFF8A5F7A954811B13CE34D4F75144D/.tez/application_1463524588189_0001
2016-05-17 22:36:28,548 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_ALL/_temporary
2016-05-17 22:36:28,549 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.390
2016-05-17 22:36:28,627 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_'field'_ALL
2016-05-17 22:36:28,654 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:28,654 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:28,692 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.007
2016-05-17 22:36:28,697 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.012
2016-05-17 22:36:28,697 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:28,698 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:28,698 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_'field'_REPLACE
2016-05-17 22:36:28,725 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:28,725 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:28,774 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-17 22:36:28,783 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-05-17 22:36:28,783 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:28,784 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:28,784 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:28,801 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:28,802 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_SWAP
2016-05-17 22:36:28,814 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:28,814 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:28,814 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_SWAP"]
2016-05-17 22:36:28,815 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:28,815 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:28,816 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:28,817 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...yplatform/-1_'field'_SWAP
2016-05-17 22:36:28,817 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:28,830 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:28,835 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:28,835 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:28,835 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:28,845 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B94D472C3B634F03B2906F0A1EE5731F/DC668609793641ADB41A0C26549246D7/.tez/application_1463524588835_0001 doesn't exist and is created
2016-05-17 22:36:28,863 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:28,863 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:28,863 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B94D472C3B634F03B2906F0A1EE5731F/DC668609793641ADB41A0C26549246D7/.tez/application_1463524588835_0001
2016-05-17 22:36:28,868 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524588835_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:28,880 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:28,881 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:28,883 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:28,883 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:28,884 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B94D472C3B634F03B2906F0A1EE5731F/DC668609793641ADB41A0C26549246D7/.tez/application_1463524588835_0001/localmode-log-dir
2016-05-17 22:36:28,884 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B94D472C3B634F03B2906F0A1EE5731F/DC668609793641ADB41A0C26549246D7/.tez/application_1463524588835_0001/localmode-log-dir/history.txt.appattempt_1463524588835_0001_000000, maxErrors=10
2016-05-17 22:36:28,884 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:28,884 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524588835_0001_000000, appSubmitTime=1463524588867, launchTime=1463524588867
2016-05-17 22:36:28,892 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:28,895 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49109
2016-05-17 22:36:28,895 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:28,901 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:28,902 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:28,902 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:28,905 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:28,905 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524588835_0001_000000, startTime=1463524588905
2016-05-17 22:36:28,906 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:28,963 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:28,964 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:28,964 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/-1_'field'_SWAP, applicationId=application_1463524588835_0001, dagName=[B94D472C3B634F03B2906F0A1EE5731F/DC668609793641ADB41A0C26549246D7] each/(1/1) ...yplatform/-1_'field'_SWAP
2016-05-17 22:36:28,984 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [B94D472C3B634F03B2906F0A1EE5731F/DC668609793641ADB41A0C26549246D7] each/(1/1) ...yplatform/-1_'field'_SWAP
2016-05-17 22:36:28,986 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524588835_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B94D472C3B634F03B2906F0A1EE5731F/DC668609793641ADB41A0C26549246D7/.tez/application_1463524588835_0001/localmode-log-dir/dag_1463524588835_0001_1.dot
2016-05-17 22:36:28,987 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B94D472C3B634F03B2906F0A1EE5731F/DC668609793641ADB41A0C26549246D7/.tez/application_1463524588835_0001/localmode-log-dir/dag_1463524588835_0001_1-tez-dag.pb.txt
2016-05-17 22:36:29,004 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524588835_0001_1
2016-05-17 22:36:29,004 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:29,004 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [B94D472C3B634F03B2906F0A1EE5731F/DC668609793641ADB41A0C26549246D7] each/(1/1) ...yplatform/-1_'field'_SWAP
2016-05-17 22:36:29 Running Dag: dag_1463524588835_0001_1
2016-05-17 22:36:29,018 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588835_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524588835_0001_1, submitTime=1463524588984
2016-05-17 22:36:29,018 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:29,018 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CA4BB83BE24A42FB949EB2A65011B806
2016-05-17 22:36:29,019 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:29,019 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:29,020 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588835_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524588835_0001_1, initTime=1463524589018
2016-05-17 22:36:29,020 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524588835_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:29,021 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588835_0001_1][Event:DAG_STARTED]: dagID=dag_1463524588835_0001_1, startTime=1463524589021
2016-05-17 22:36:29,021 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:29,021 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524588835_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:29,021 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CA4BB83BE24A42FB949EB2A65011B806 : {3DF1253666B94A9085E6DA29D607CD83={InputName=3DF1253666B94A9085E6DA29D607CD83}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:29,022 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 3DF1253666B94A9085E6DA29D607CD83, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:29,022 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806]
2016-05-17 22:36:29,021 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:29,022 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806]
2016-05-17 22:36:29,022 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806]
2016-05-17 22:36:29,023 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806]. Starting root input initializers: 1
2016-05-17 22:36:29,023 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/-1_'field'_SWAP, applicationId=application_1463524588835_0001, dagName=[B94D472C3B634F03B2906F0A1EE5731F/DC668609793641ADB41A0C26549246D7] each/(1/1) ...yplatform/-1_'field'_SWAP
2016-05-17 22:36:29,023 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:29,024 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 3DF1253666B94A9085E6DA29D607CD83 on vertex vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806]
2016-05-17 22:36:29,026 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:29,026 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 3DF1253666B94A9085E6DA29D607CD83 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:29,028 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:29,028 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:29,030 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:29,031 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 3DF1253666B94A9085E6DA29D607CD83 on vertex vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806]
2016-05-17 22:36:29,032 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:29,032 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {3DF1253666B94A9085E6DA29D607CD83=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:29,032 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806] parallelism set to 1
2016-05-17 22:36:29,032 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:29,033 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588835_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524588835_0001_1_00, eventCount=1
2016-05-17 22:36:29,033 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806]
2016-05-17 22:36:29,034 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=F77D6D3AFBCD4019905CB3B96283B846, vertexId=vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:29,034 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=F77D6D3AFBCD4019905CB3B96283B846, vertexId=vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806]
2016-05-17 22:36:29,036 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=F77D6D3AFBCD4019905CB3B96283B846, vertexName=CA4BB83BE24A42FB949EB2A65011B806, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:29,037 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=F77D6D3AFBCD4019905CB3B96283B846, vertexId=vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806]
2016-05-17 22:36:29,038 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:29,039 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588835_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CA4BB83BE24A42FB949EB2A65011B806, vertexId=vertex_1463524588835_0001_1_00, initRequestedTime=1463524589021, initedTime=1463524589038, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:29,039 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806] with distanceFromRoot: 0
2016-05-17 22:36:29,039 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:29,040 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CA4BB83BE24A42FB949EB2A65011B806
2016-05-17 22:36:29,040 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806]
2016-05-17 22:36:29,041 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:29,046 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588835_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524588835_0001_1_00, startRequestedTime=1463524589040, startedTime=1463524589040
2016-05-17 22:36:29,046 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:29,047 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:29,048 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588835_0001_1][Event:TASK_STARTED]: vertexName=CA4BB83BE24A42FB949EB2A65011B806, taskId=task_1463524588835_0001_1_00_000000, scheduledTime=1463524589047, launchTime=1463524589047
2016-05-17 22:36:29,049 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524588835_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:29,049 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806]
2016-05-17 22:36:29,049 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B94D472C3B634F03B2906F0A1EE5731F/DC668609793641ADB41A0C26549246D7] each/(1/1) ...yplatform/-1_'field'_SWAP, VertexName: CA4BB83BE24A42FB949EB2A65011B806, VertexParallelism: 1, TaskAttemptID:attempt_1463524588835_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3DF1253666B94A9085E6DA29D607CD83, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F77D6D3AFBCD4019905CB3B96283B846, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:29,049 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524588835_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:29,050 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:29,053 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:29,054 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:29,055 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:29,055 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524588835_0001, with dag id: dag_1463524588835_0001_1
2016-05-17 22:36:29,055 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524588835_0001_00_000001
2016-05-17 22:36:29,057 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524588835_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:29,061 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524588835_0001_00_000001
2016-05-17 22:36:29,061 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:29,064 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524588835_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:29,064 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:29,065 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588835_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524588835_0001_00_000001, launchTime=1463524589063
2016-05-17 22:36:29,066 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524588835_0001_00_000001
2016-05-17 22:36:29,068 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524588835_0001_1_00_000000_0] to container: [container_1463524588835_0001_00_000001]
2016-05-17 22:36:29,069 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524588835_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:29,069 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524588835_0001_1_00_000000_0] started. Is using containerId: [container_1463524588835_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:29,070 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588835_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CA4BB83BE24A42FB949EB2A65011B806, taskAttemptId=attempt_1463524588835_0001_1_00_000000_0, startTime=1463524589069, containerId=container_1463524588835_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524588835_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:29,070 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524588835_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:29,070 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524588835_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:29,070 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:29,074 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524588835_0001_00_000001 given task: attempt_1463524588835_0001_1_00_000000_0
2016-05-17 22:36:29,074 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524588835_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524588835_0001_1_00_000000_0
2016-05-17 22:36:29 Starting to run new task attempt: attempt_1463524588835_0001_1_00_000000_0
2016-05-17 22:36:29,075 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524588835_0001_1_00_000000_0
2016-05-17 22:36:29,075 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:29,075 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:29,076 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:29,076 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:29,076 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:29,076 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:29,076 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B94D472C3B634F03B2906F0A1EE5731F/DC668609793641ADB41A0C26549246D7] each/(1/1) ...yplatform/-1_'field'_SWAP, VertexName: CA4BB83BE24A42FB949EB2A65011B806, VertexParallelism: 1, TaskAttemptID:attempt_1463524588835_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3DF1253666B94A9085E6DA29D607CD83, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F77D6D3AFBCD4019905CB3B96283B846, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:29,076 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:29,078 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524588835_0001_1_00_000000_0
2016-05-17 22:36:29,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:29,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:29,083 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524588835_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:29,085 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=3DF1253666B94A9085E6DA29D607CD83, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:29,085 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:29,086 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 3DF1253666B94A9085E6DA29D607CD83
2016-05-17 22:36:29,086 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 3DF1253666B94A9085E6DA29D607CD83
2016-05-17 22:36:29,087 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:29,087 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F77D6D3AFBCD4019905CB3B96283B846, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:29,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:29,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F77D6D3AFBCD4019905CB3B96283B846
2016-05-17 22:36:29,088 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:29,088 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: F77D6D3AFBCD4019905CB3B96283B846
2016-05-17 22:36:29,091 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:29,094 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:29,096 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 3DF1253666B94A9085E6DA29D607CD83
2016-05-17 22:36:29,096 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 3DF1253666B94A9085E6DA29D607CD83
2016-05-17 22:36:29,104 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CA4BB83BE24A42FB949EB2A65011B806, ordinal: 0
2016-05-17 22:36:29,104 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CA4BB83BE24A42FB949EB2A65011B806, mem on start (mb), free: 246, total: 389, max: 672
2016-05-17 22:36:29,104 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:29,108 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:29,109 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F77D6D3AFBCD4019905CB3B96283B846
2016-05-17 22:36:29,109 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:29,109 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:29,109 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:29,110 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:29,110 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:29,110 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:29,111 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:29,111 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:29,111 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:29,112 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:29,112 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 3DF1253666B94A9085E6DA29D607CD83, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:29,112 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F77D6D3AFBCD4019905CB3B96283B846, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:29,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:29,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 3DF1253666B94A9085E6DA29D607CD83 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:29,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 3DF1253666B94A9085E6DA29D607CD83
2016-05-17 22:36:29,113 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:29,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 3DF1253666B94A9085E6DA29D607CD83
2016-05-17 22:36:29,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:29,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:29,115 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:29,115 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524588835_0001_1_00_000000_0
2016-05-17 22:36:29,119 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:29,119 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:29,119 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@58e04da3
2016-05-17 22:36:29,120 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 3DF1253666B94A9085E6DA29D607CD83
2016-05-17 22:36:29,120 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_SWAP"], id: F77D6D3AFBCD4019905CB3B96283B846
2016-05-17 22:36:29,120 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_SWAP"]
2016-05-17 22:36:29,121 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:29,121 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:29,121 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@58e04da3
2016-05-17 22:36:29,122 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:29,122 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:29,122 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CA4BB83BE24A42FB949EB2A65011B806, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:29,124 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:29,125 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:29,125 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524588835_0001_1_00_000000_0
2016-05-17 22:36:29,125 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524588835_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:29,126 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524588835_0001_1_00_000000_0
2016-05-17 22:36:29,126 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CA4BB83BE24A42FB949EB2A65011B806, mem on close (mb), free: 244, total: 389, max: 672
2016-05-17 22:36:29 Completed running task attempt: attempt_1463524588835_0001_1_00_000000_0
2016-05-17 22:36:29,127 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524588835_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:29,128 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=408420352, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524589122, Process_Duration=4, Process_End_Time=1463524589126, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:29,129 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:29,129 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:29,129 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588835_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CA4BB83BE24A42FB949EB2A65011B806, taskAttemptId=attempt_1463524588835_0001_1_00_000000_0, startTime=1463524589069, finishTime=1463524589128, timeTaken=59, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=408420352, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524589122, Process_Duration=4, Process_End_Time=1463524589126, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:29,130 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524588835_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:29,130 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:29,129 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:29,131 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524588835_0001_00_000001
2016-05-17 22:36:29,130 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524588835_0001_1_00_000000_0
2016-05-17 22:36:29,132 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588835_0001_1][Event:TASK_FINISHED]: vertexName=CA4BB83BE24A42FB949EB2A65011B806, taskId=task_1463524588835_0001_1_00_000000, startTime=1463524589069, finishTime=1463524589132, timeTaken=63, status=SUCCEEDED, successfulAttemptID=attempt_1463524588835_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=408420352, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524589122, Process_Duration=4, Process_End_Time=1463524589126, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:29,132 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524588835_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:29,133 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806] : 1
2016-05-17 22:36:29,133 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806]
2016-05-17 22:36:29,134 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:29,135 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:29,136 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588835_0001_1][Event:VERTEX_FINISHED]: vertexName=CA4BB83BE24A42FB949EB2A65011B806, vertexId=vertex_1463524588835_0001_1_00, initRequestedTime=1463524589021, initedTime=1463524589038, startRequestedTime=1463524589040, startedTime=1463524589040, finishTime=1463524589133, timeTaken=93, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=408420352, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524589122, Process_Duration=4, Process_End_Time=1463524589126, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524589069, firstTasksToStart=[ task_1463524588835_0001_1_00_000000 ], lastTaskFinishTime=1463524589128, lastTasksToFinish=[ task_1463524588835_0001_1_00_000000 ], minTaskDuration=59, maxTaskDuration=59, avgTaskDuration=59.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524588835_0001_1_00_000000 ], longestDurationTasks=[ task_1463524588835_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:29,136 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:29,136 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524588835_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:29,137 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588835_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524588835_0001_00_000001, stoppedTime=1463524589137, exitStatus=0
2016-05-17 22:36:29,137 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524588835_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:29,137 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:29,137 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524588835_0001_1
2016-05-17 22:36:29,137 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588835_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524588835_0001_1
2016-05-17 22:36:29,138 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: F77D6D3AFBCD4019905CB3B96283B846 for vertex: vertex_1463524588835_0001_1_00 [CA4BB83BE24A42FB949EB2A65011B806]
2016-05-17 22:36:29,138 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:29,142 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524588835_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:29,143 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524588835_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:29,143 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524588835_0001_00_000001, Exiting
2016-05-17 22:36:29,143 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524588835_0001_00_000001
2016-05-17 22:36:29,143 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524588835_0001_00_000001
2016-05-17 22:36:29,146 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524588835_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:29,146 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524588835_0001_00_000001 completed successfully
2016-05-17 22:36:29,148 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524588835_0001_1, queueSize=0
2016-05-17 22:36:29,148 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:29,148 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524588835_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524588835_0001_1, startTime=1463524589021, finishTime=1463524589146, timeTaken=125, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=408420352, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524589122, Process_Duration=4, Process_End_Time=1463524589126, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:29,148 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524588835_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:29,149 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524588835_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:29,149 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524588835_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:29 Completed Dag: dag_1463524588835_0001_1
2016-05-17 22:36:29,149 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524588835_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:29,149 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524588835_0001_1_post
2016-05-17 22:36:29,149 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:29,149 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:29,150 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524588835_0001_00_000001
2016-05-17 22:36:29,157 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/-1_'field'_SWAP, applicationId=application_1463524588835_0001
2016-05-17 22:36:29,159 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:29,159 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:29,160 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:29,160 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:29,160 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:29,160 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:29,161 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:29,161 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:29,161 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:29,161 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:29,162 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:29,162 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:29,162 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:29,167 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B94D472C3B634F03B2906F0A1EE5731F/DC668609793641ADB41A0C26549246D7/.tez/application_1463524588835_0001
2016-05-17 22:36:29,167 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_'field'_SWAP/_temporary
2016-05-17 22:36:29,168 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.353
2016-05-17 22:36:29,255 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_'field'_SWAP
2016-05-17 22:36:29,292 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:29,292 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:29,348 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 22:36:29,361 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-05-17 22:36:29,362 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:29,362 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:29,362 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:29,384 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:29,385 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_-1
2016-05-17 22:36:29,399 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:29,400 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:29,400 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_-1"]
2016-05-17 22:36:29,400 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:29,401 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:29,401 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:29,402 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...blyplatform/-1_UNKNOWN_-1
2016-05-17 22:36:29,402 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:29,420 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:29,426 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:29,427 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:29,427 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:29,437 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CDDEAE911BE143A6B8DF99937D18CA97/C79246B258624D3DAD7E214A73DCB3AA/.tez/application_1463524589427_0001 doesn't exist and is created
2016-05-17 22:36:29,457 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:29,457 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:29,458 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CDDEAE911BE143A6B8DF99937D18CA97/C79246B258624D3DAD7E214A73DCB3AA/.tez/application_1463524589427_0001
2016-05-17 22:36:29,463 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524589427_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:29,480 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:29,481 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:29,483 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:29,483 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:29,483 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CDDEAE911BE143A6B8DF99937D18CA97/C79246B258624D3DAD7E214A73DCB3AA/.tez/application_1463524589427_0001/localmode-log-dir
2016-05-17 22:36:29,484 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CDDEAE911BE143A6B8DF99937D18CA97/C79246B258624D3DAD7E214A73DCB3AA/.tez/application_1463524589427_0001/localmode-log-dir/history.txt.appattempt_1463524589427_0001_000000, maxErrors=10
2016-05-17 22:36:29,484 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:29,484 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524589427_0001_000000, appSubmitTime=1463524589462, launchTime=1463524589462
2016-05-17 22:36:29,489 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:29,491 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:29,493 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:29,493 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:29,494 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:57685
2016-05-17 22:36:29,497 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:29,501 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524589427_0001_000000, startTime=1463524589501
2016-05-17 22:36:29,501 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:29,501 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:29,558 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:29,558 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:29,558 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/-1_UNKNOWN_-1, applicationId=application_1463524589427_0001, dagName=[CDDEAE911BE143A6B8DF99937D18CA97/C79246B258624D3DAD7E214A73DCB3AA] each/(1/1) ...blyplatform/-1_UNKNOWN_-1
2016-05-17 22:36:29,584 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [CDDEAE911BE143A6B8DF99937D18CA97/C79246B258624D3DAD7E214A73DCB3AA] each/(1/1) ...blyplatform/-1_UNKNOWN_-1
2016-05-17 22:36:29,586 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524589427_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CDDEAE911BE143A6B8DF99937D18CA97/C79246B258624D3DAD7E214A73DCB3AA/.tez/application_1463524589427_0001/localmode-log-dir/dag_1463524589427_0001_1.dot
2016-05-17 22:36:29,587 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CDDEAE911BE143A6B8DF99937D18CA97/C79246B258624D3DAD7E214A73DCB3AA/.tez/application_1463524589427_0001/localmode-log-dir/dag_1463524589427_0001_1-tez-dag.pb.txt
2016-05-17 22:36:29,620 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524589427_0001_1
2016-05-17 22:36:29,620 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:29,621 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [CDDEAE911BE143A6B8DF99937D18CA97/C79246B258624D3DAD7E214A73DCB3AA] each/(1/1) ...blyplatform/-1_UNKNOWN_-1
2016-05-17 22:36:29 Running Dag: dag_1463524589427_0001_1
2016-05-17 22:36:29,644 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524589427_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524589427_0001_1, submitTime=1463524589584
2016-05-17 22:36:29,644 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:29,646 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A317374FC3844A828D729731E85305E7
2016-05-17 22:36:29,646 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:29,647 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524589427_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524589427_0001_1, initTime=1463524589644
2016-05-17 22:36:29,647 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:29,649 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524589427_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:29,651 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524589427_0001_1][Event:DAG_STARTED]: dagID=dag_1463524589427_0001_1, startTime=1463524589651
2016-05-17 22:36:29,651 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:29,651 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524589427_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:29,652 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A317374FC3844A828D729731E85305E7 : {2B9CD18D70FB469A96D65D28B890EBE6={InputName=2B9CD18D70FB469A96D65D28B890EBE6}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:29,651 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:29,652 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 2B9CD18D70FB469A96D65D28B890EBE6, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:29,652 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7]
2016-05-17 22:36:29,652 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7]
2016-05-17 22:36:29,652 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7]
2016-05-17 22:36:29,652 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/-1_UNKNOWN_-1, applicationId=application_1463524589427_0001, dagName=[CDDEAE911BE143A6B8DF99937D18CA97/C79246B258624D3DAD7E214A73DCB3AA] each/(1/1) ...blyplatform/-1_UNKNOWN_-1
2016-05-17 22:36:29,653 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7]. Starting root input initializers: 1
2016-05-17 22:36:29,653 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:29,667 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 2B9CD18D70FB469A96D65D28B890EBE6 on vertex vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7]
2016-05-17 22:36:29,670 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:29,670 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 2B9CD18D70FB469A96D65D28B890EBE6 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:29,671 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:29,671 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:29,674 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:29,676 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 2B9CD18D70FB469A96D65D28B890EBE6 on vertex vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7]
2016-05-17 22:36:29,676 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:29,676 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {2B9CD18D70FB469A96D65D28B890EBE6=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:29,676 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7] parallelism set to 1
2016-05-17 22:36:29,677 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:29,677 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524589427_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524589427_0001_1_00, eventCount=1
2016-05-17 22:36:29,677 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7]
2016-05-17 22:36:29,677 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=CAAE0FAEEAE4414EA3998E8AF2C530EB, vertexId=vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:29,678 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=CAAE0FAEEAE4414EA3998E8AF2C530EB, vertexId=vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7]
2016-05-17 22:36:29,683 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=CAAE0FAEEAE4414EA3998E8AF2C530EB, vertexName=A317374FC3844A828D729731E85305E7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:29,684 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=CAAE0FAEEAE4414EA3998E8AF2C530EB, vertexId=vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7]
2016-05-17 22:36:29,686 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:29,686 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524589427_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A317374FC3844A828D729731E85305E7, vertexId=vertex_1463524589427_0001_1_00, initRequestedTime=1463524589652, initedTime=1463524589686, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:29,687 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7] with distanceFromRoot: 0
2016-05-17 22:36:29,687 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:29,688 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A317374FC3844A828D729731E85305E7
2016-05-17 22:36:29,688 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7]
2016-05-17 22:36:29,688 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:29,689 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524589427_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524589427_0001_1_00, startRequestedTime=1463524589688, startedTime=1463524589688
2016-05-17 22:36:29,689 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:29,690 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:29,691 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524589427_0001_1][Event:TASK_STARTED]: vertexName=A317374FC3844A828D729731E85305E7, taskId=task_1463524589427_0001_1_00_000000, scheduledTime=1463524589690, launchTime=1463524589690
2016-05-17 22:36:29,691 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524589427_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:29,691 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7]
2016-05-17 22:36:29,692 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [CDDEAE911BE143A6B8DF99937D18CA97/C79246B258624D3DAD7E214A73DCB3AA] each/(1/1) ...blyplatform/-1_UNKNOWN_-1, VertexName: A317374FC3844A828D729731E85305E7, VertexParallelism: 1, TaskAttemptID:attempt_1463524589427_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2B9CD18D70FB469A96D65D28B890EBE6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CAAE0FAEEAE4414EA3998E8AF2C530EB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:29,692 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524589427_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:29,693 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:29,694 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:29,698 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:29,698 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:29,699 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524589427_0001_00_000001
2016-05-17 22:36:29,700 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524589427_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:29,701 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524589427_0001, with dag id: dag_1463524589427_0001_1
2016-05-17 22:36:29,707 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524589427_0001_00_000001
2016-05-17 22:36:29,707 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:29,709 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524589427_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:29,710 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:29,710 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524589427_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524589427_0001_00_000001, launchTime=1463524589708
2016-05-17 22:36:29,711 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524589427_0001_00_000001
2016-05-17 22:36:29,713 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524589427_0001_1_00_000000_0] to container: [container_1463524589427_0001_00_000001]
2016-05-17 22:36:29,713 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524589427_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:29,713 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524589427_0001_00_000001 given task: attempt_1463524589427_0001_1_00_000000_0
2016-05-17 22:36:29,714 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524589427_0001_1_00_000000_0] started. Is using containerId: [container_1463524589427_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:29,715 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:29,716 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524589427_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A317374FC3844A828D729731E85305E7, taskAttemptId=attempt_1463524589427_0001_1_00_000000_0, startTime=1463524589713, containerId=container_1463524589427_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524589427_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:29,714 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524589427_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524589427_0001_1_00_000000_0
2016-05-17 22:36:29,716 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524589427_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:29 Starting to run new task attempt: attempt_1463524589427_0001_1_00_000000_0
2016-05-17 22:36:29,718 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524589427_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:29,717 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524589427_0001_1_00_000000_0
2016-05-17 22:36:29,719 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:29,719 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:29,720 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:29,720 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:29,720 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:29,720 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:29,720 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [CDDEAE911BE143A6B8DF99937D18CA97/C79246B258624D3DAD7E214A73DCB3AA] each/(1/1) ...blyplatform/-1_UNKNOWN_-1, VertexName: A317374FC3844A828D729731E85305E7, VertexParallelism: 1, TaskAttemptID:attempt_1463524589427_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2B9CD18D70FB469A96D65D28B890EBE6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CAAE0FAEEAE4414EA3998E8AF2C530EB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:29,720 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:29,722 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524589427_0001_1_00_000000_0
2016-05-17 22:36:29,723 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:29,723 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:29,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=2B9CD18D70FB469A96D65D28B890EBE6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:29,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:29,726 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 2B9CD18D70FB469A96D65D28B890EBE6
2016-05-17 22:36:29,726 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 2B9CD18D70FB469A96D65D28B890EBE6
2016-05-17 22:36:29,727 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:29,730 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524589427_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:29,734 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:29,736 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CAAE0FAEEAE4414EA3998E8AF2C530EB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:29,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:29,740 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CAAE0FAEEAE4414EA3998E8AF2C530EB
2016-05-17 22:36:29,741 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:29,742 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: CAAE0FAEEAE4414EA3998E8AF2C530EB
2016-05-17 22:36:29,746 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:29,748 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 2B9CD18D70FB469A96D65D28B890EBE6
2016-05-17 22:36:29,748 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 2B9CD18D70FB469A96D65D28B890EBE6
2016-05-17 22:36:29,749 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A317374FC3844A828D729731E85305E7, ordinal: 0
2016-05-17 22:36:29,749 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A317374FC3844A828D729731E85305E7, mem on start (mb), free: 165, total: 389, max: 672
2016-05-17 22:36:29,750 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:29,750 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:29,751 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:29,764 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:29,764 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CAAE0FAEEAE4414EA3998E8AF2C530EB
2016-05-17 22:36:29,765 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:29,765 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:29,765 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:29,765 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:29,765 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:29,766 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:29,766 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:29,766 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:29,766 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 2B9CD18D70FB469A96D65D28B890EBE6, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:29,766 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CAAE0FAEEAE4414EA3998E8AF2C530EB, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:29,766 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:29,767 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 2B9CD18D70FB469A96D65D28B890EBE6 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:29,767 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:29,767 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:29,767 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 2B9CD18D70FB469A96D65D28B890EBE6
2016-05-17 22:36:29,767 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:29,767 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 2B9CD18D70FB469A96D65D28B890EBE6
2016-05-17 22:36:29,768 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:29,768 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524589427_0001_1_00_000000_0
2016-05-17 22:36:29,774 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:29,775 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:29,775 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 2B9CD18D70FB469A96D65D28B890EBE6
2016-05-17 22:36:29,776 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_-1"], id: CAAE0FAEEAE4414EA3998E8AF2C530EB
2016-05-17 22:36:29,776 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_-1"]
2016-05-17 22:36:29,777 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:29,777 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:29,775 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2588015b
2016-05-17 22:36:29,779 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2588015b
2016-05-17 22:36:29,780 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:29,780 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:29,780 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A317374FC3844A828D729731E85305E7, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:29,787 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:29,789 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:29,789 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524589427_0001_1_00_000000_0
2016-05-17 22:36:29,790 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524589427_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:29,792 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524589427_0001_1_00_000000_0
2016-05-17 22:36:29,792 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A317374FC3844A828D729731E85305E7, mem on close (mb), free: 163, total: 389, max: 672
2016-05-17 22:36:29 Completed running task attempt: attempt_1463524589427_0001_1_00_000000_0
2016-05-17 22:36:29,795 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524589427_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:29,797 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=408420352, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524589780, Process_Duration=12, Process_End_Time=1463524589792, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:29,798 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:29,798 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:29,798 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524589427_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A317374FC3844A828D729731E85305E7, taskAttemptId=attempt_1463524589427_0001_1_00_000000_0, startTime=1463524589713, finishTime=1463524589797, timeTaken=84, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=408420352, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524589780, Process_Duration=12, Process_End_Time=1463524589792, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:29,799 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524589427_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:29,799 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524589427_0001_1_00_000000_0
2016-05-17 22:36:29,799 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:29,801 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524589427_0001_00_000001
2016-05-17 22:36:29,801 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:29,802 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524589427_0001_1][Event:TASK_FINISHED]: vertexName=A317374FC3844A828D729731E85305E7, taskId=task_1463524589427_0001_1_00_000000, startTime=1463524589713, finishTime=1463524589799, timeTaken=86, status=SUCCEEDED, successfulAttemptID=attempt_1463524589427_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=408420352, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524589780, Process_Duration=12, Process_End_Time=1463524589792, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:29,802 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524589427_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:29,802 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:29,802 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7] : 1
2016-05-17 22:36:29,802 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7]
2016-05-17 22:36:29,804 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:29,804 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524589427_0001_1][Event:VERTEX_FINISHED]: vertexName=A317374FC3844A828D729731E85305E7, vertexId=vertex_1463524589427_0001_1_00, initRequestedTime=1463524589652, initedTime=1463524589686, startRequestedTime=1463524589688, startedTime=1463524589688, finishTime=1463524589802, timeTaken=114, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=408420352, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524589780, Process_Duration=12, Process_End_Time=1463524589792, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524589713, firstTasksToStart=[ task_1463524589427_0001_1_00_000000 ], lastTaskFinishTime=1463524589797, lastTasksToFinish=[ task_1463524589427_0001_1_00_000000 ], minTaskDuration=84, maxTaskDuration=84, avgTaskDuration=84.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524589427_0001_1_00_000000 ], longestDurationTasks=[ task_1463524589427_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:29,805 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:29,805 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524589427_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:29,805 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524589427_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524589427_0001_00_000001, stoppedTime=1463524589805, exitStatus=0
2016-05-17 22:36:29,805 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:29,805 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524589427_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:29,805 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:29,806 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524589427_0001_1
2016-05-17 22:36:29,806 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524589427_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524589427_0001_1
2016-05-17 22:36:29,806 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: CAAE0FAEEAE4414EA3998E8AF2C530EB for vertex: vertex_1463524589427_0001_1_00 [A317374FC3844A828D729731E85305E7]
2016-05-17 22:36:29,811 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524589427_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:29,811 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524589427_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:29,812 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524589427_0001_00_000001, Exiting
2016-05-17 22:36:29,812 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524589427_0001_00_000001
2016-05-17 22:36:29,812 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524589427_0001_00_000001
2016-05-17 22:36:29,813 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524589427_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:29,814 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524589427_0001_00_000001 completed successfully
2016-05-17 22:36:29,819 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524589427_0001_1, queueSize=0
2016-05-17 22:36:29,819 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524589427_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524589427_0001_1, startTime=1463524589651, finishTime=1463524589818, timeTaken=167, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=408420352, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524589780, Process_Duration=12, Process_End_Time=1463524589792, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:29,819 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524589427_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:29,820 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524589427_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:29,820 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:29,820 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524589427_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:29 Completed Dag: dag_1463524589427_0001_1
2016-05-17 22:36:29,820 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524589427_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:29,821 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524589427_0001_1_post
2016-05-17 22:36:29,821 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:29,821 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:29,821 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524589427_0001_00_000001
2016-05-17 22:36:29,828 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/-1_UNKNOWN_-1, applicationId=application_1463524589427_0001
2016-05-17 22:36:29,833 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:29,833 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:29,833 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:29,833 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:29,833 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:29,834 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:29,834 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:29,834 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:29,835 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:29,835 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:29,834 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:29,835 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:29,836 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:29,844 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CDDEAE911BE143A6B8DF99937D18CA97/C79246B258624D3DAD7E214A73DCB3AA/.tez/application_1463524589427_0001
2016-05-17 22:36:29,845 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_-1/_temporary
2016-05-17 22:36:29,845 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.445
2016-05-17 22:36:30,000 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_UNKNOWN_-1
2016-05-17 22:36:30,028 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:30,028 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:30,065 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.009
2016-05-17 22:36:30,068 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.012
2016-05-17 22:36:30,069 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:30,069 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:30,069 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_UNKNOWN_'field'
2016-05-17 22:36:30,096 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:30,096 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:30,142 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-17 22:36:30,152 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.028
2016-05-17 22:36:30,153 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:30,153 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:30,153 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:30,171 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:30,172 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_RESULTS
2016-05-17 22:36:30,187 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:30,188 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:30,188 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_RESULTS"]
2016-05-17 22:36:30,189 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:30,190 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:30,190 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:30,191 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...atform/-1_UNKNOWN_RESULTS
2016-05-17 22:36:30,191 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:30,205 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:30,210 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:30,211 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:30,211 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:30,222 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4BF9BD2E29E140B78FCFD374E8FC0162/D15B4C66AAEF45A9886AA1D521D4E7C3/.tez/application_1463524590211_0001 doesn't exist and is created
2016-05-17 22:36:30,241 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:30,241 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:30,242 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4BF9BD2E29E140B78FCFD374E8FC0162/D15B4C66AAEF45A9886AA1D521D4E7C3/.tez/application_1463524590211_0001
2016-05-17 22:36:30,247 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524590211_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:30,260 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:30,261 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:30,264 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:30,264 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:30,264 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4BF9BD2E29E140B78FCFD374E8FC0162/D15B4C66AAEF45A9886AA1D521D4E7C3/.tez/application_1463524590211_0001/localmode-log-dir
2016-05-17 22:36:30,265 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4BF9BD2E29E140B78FCFD374E8FC0162/D15B4C66AAEF45A9886AA1D521D4E7C3/.tez/application_1463524590211_0001/localmode-log-dir/history.txt.appattempt_1463524590211_0001_000000, maxErrors=10
2016-05-17 22:36:30,265 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:30,266 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524590211_0001_000000, appSubmitTime=1463524590246, launchTime=1463524590246
2016-05-17 22:36:30,273 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:30,281 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:30,282 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:30,283 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:30,289 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:33351
2016-05-17 22:36:30,290 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:30,295 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524590211_0001_000000, startTime=1463524590295
2016-05-17 22:36:30,295 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:30,295 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:30,341 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:30,342 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:30,342 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/-1_UNKNOWN_RESULTS, applicationId=application_1463524590211_0001, dagName=[4BF9BD2E29E140B78FCFD374E8FC0162/D15B4C66AAEF45A9886AA1D521D4E7C3] each/(1/1) ...atform/-1_UNKNOWN_RESULTS
2016-05-17 22:36:30,369 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [4BF9BD2E29E140B78FCFD374E8FC0162/D15B4C66AAEF45A9886AA1D521D4E7C3] each/(1/1) ...atform/-1_UNKNOWN_RESULTS
2016-05-17 22:36:30,372 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524590211_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4BF9BD2E29E140B78FCFD374E8FC0162/D15B4C66AAEF45A9886AA1D521D4E7C3/.tez/application_1463524590211_0001/localmode-log-dir/dag_1463524590211_0001_1.dot
2016-05-17 22:36:30,374 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4BF9BD2E29E140B78FCFD374E8FC0162/D15B4C66AAEF45A9886AA1D521D4E7C3/.tez/application_1463524590211_0001/localmode-log-dir/dag_1463524590211_0001_1-tez-dag.pb.txt
2016-05-17 22:36:30,387 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524590211_0001_1
2016-05-17 22:36:30,387 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:30,388 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [4BF9BD2E29E140B78FCFD374E8FC0162/D15B4C66AAEF45A9886AA1D521D4E7C3] each/(1/1) ...atform/-1_UNKNOWN_RESULTS
2016-05-17 22:36:30 Running Dag: dag_1463524590211_0001_1
2016-05-17 22:36:30,406 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:30,407 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590211_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524590211_0001_1, submitTime=1463524590370
2016-05-17 22:36:30,407 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B6024A6439EE4A71BBE9884FF4997A06
2016-05-17 22:36:30,407 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:30,409 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:30,409 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590211_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524590211_0001_1, initTime=1463524590407
2016-05-17 22:36:30,409 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524590211_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:30,410 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590211_0001_1][Event:DAG_STARTED]: dagID=dag_1463524590211_0001_1, startTime=1463524590410
2016-05-17 22:36:30,410 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:30,410 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:30,410 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524590211_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:30,410 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B6024A6439EE4A71BBE9884FF4997A06 : {8176ECEC23B44B96B4E69B168A5EE793={InputName=8176ECEC23B44B96B4E69B168A5EE793}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:30,411 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 8176ECEC23B44B96B4E69B168A5EE793, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:30,411 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06]
2016-05-17 22:36:30,411 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06]
2016-05-17 22:36:30,411 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06]
2016-05-17 22:36:30,411 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06]. Starting root input initializers: 1
2016-05-17 22:36:30,412 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:30,414 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/-1_UNKNOWN_RESULTS, applicationId=application_1463524590211_0001, dagName=[4BF9BD2E29E140B78FCFD374E8FC0162/D15B4C66AAEF45A9886AA1D521D4E7C3] each/(1/1) ...atform/-1_UNKNOWN_RESULTS
2016-05-17 22:36:30,415 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 8176ECEC23B44B96B4E69B168A5EE793 on vertex vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06]
2016-05-17 22:36:30,417 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:30,417 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 8176ECEC23B44B96B4E69B168A5EE793 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:30,418 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:30,418 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:30,420 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:30,423 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 8176ECEC23B44B96B4E69B168A5EE793 on vertex vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06]
2016-05-17 22:36:30,423 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:30,423 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {8176ECEC23B44B96B4E69B168A5EE793=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:30,424 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06] parallelism set to 1
2016-05-17 22:36:30,424 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:30,424 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590211_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524590211_0001_1_00, eventCount=1
2016-05-17 22:36:30,424 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06]
2016-05-17 22:36:30,424 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D845478A286243E0895D1C279E0D1DF5, vertexId=vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:30,425 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D845478A286243E0895D1C279E0D1DF5, vertexId=vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06]
2016-05-17 22:36:30,427 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D845478A286243E0895D1C279E0D1DF5, vertexName=B6024A6439EE4A71BBE9884FF4997A06, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:30,427 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D845478A286243E0895D1C279E0D1DF5, vertexId=vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06]
2016-05-17 22:36:30,429 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:30,429 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590211_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B6024A6439EE4A71BBE9884FF4997A06, vertexId=vertex_1463524590211_0001_1_00, initRequestedTime=1463524590410, initedTime=1463524590429, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:30,429 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06] with distanceFromRoot: 0
2016-05-17 22:36:30,429 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:30,430 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B6024A6439EE4A71BBE9884FF4997A06
2016-05-17 22:36:30,430 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06]
2016-05-17 22:36:30,432 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:30,432 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590211_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524590211_0001_1_00, startRequestedTime=1463524590430, startedTime=1463524590430
2016-05-17 22:36:30,433 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:30,433 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:30,434 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590211_0001_1][Event:TASK_STARTED]: vertexName=B6024A6439EE4A71BBE9884FF4997A06, taskId=task_1463524590211_0001_1_00_000000, scheduledTime=1463524590433, launchTime=1463524590433
2016-05-17 22:36:30,434 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524590211_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:30,434 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06]
2016-05-17 22:36:30,435 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [4BF9BD2E29E140B78FCFD374E8FC0162/D15B4C66AAEF45A9886AA1D521D4E7C3] each/(1/1) ...atform/-1_UNKNOWN_RESULTS, VertexName: B6024A6439EE4A71BBE9884FF4997A06, VertexParallelism: 1, TaskAttemptID:attempt_1463524590211_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8176ECEC23B44B96B4E69B168A5EE793, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D845478A286243E0895D1C279E0D1DF5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:30,435 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524590211_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:30,435 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:30,438 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:30,438 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:30,439 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:30,439 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524590211_0001_00_000001
2016-05-17 22:36:30,439 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524590211_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:30,441 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524590211_0001_00_000001
2016-05-17 22:36:30,442 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:30,443 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524590211_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:30,443 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:30,444 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590211_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524590211_0001_00_000001, launchTime=1463524590443
2016-05-17 22:36:30,444 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524590211_0001_00_000001
2016-05-17 22:36:30,446 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524590211_0001_1_00_000000_0] to container: [container_1463524590211_0001_00_000001]
2016-05-17 22:36:30,446 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524590211_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:30,447 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524590211_0001_1_00_000000_0] started. Is using containerId: [container_1463524590211_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:30,447 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590211_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B6024A6439EE4A71BBE9884FF4997A06, taskAttemptId=attempt_1463524590211_0001_1_00_000000_0, startTime=1463524590447, containerId=container_1463524590211_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524590211_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:30,448 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524590211_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:30,448 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524590211_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:30,448 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:30,449 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524590211_0001_00_000001 given task: attempt_1463524590211_0001_1_00_000000_0
2016-05-17 22:36:30,449 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524590211_0001, with dag id: dag_1463524590211_0001_1
2016-05-17 22:36:30,449 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524590211_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524590211_0001_1_00_000000_0
2016-05-17 22:36:30 Starting to run new task attempt: attempt_1463524590211_0001_1_00_000000_0
2016-05-17 22:36:30,450 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524590211_0001_1_00_000000_0
2016-05-17 22:36:30,451 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:30,451 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:30,451 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:30,451 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:30,451 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:30,451 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:30,451 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [4BF9BD2E29E140B78FCFD374E8FC0162/D15B4C66AAEF45A9886AA1D521D4E7C3] each/(1/1) ...atform/-1_UNKNOWN_RESULTS, VertexName: B6024A6439EE4A71BBE9884FF4997A06, VertexParallelism: 1, TaskAttemptID:attempt_1463524590211_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8176ECEC23B44B96B4E69B168A5EE793, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D845478A286243E0895D1C279E0D1DF5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:30,452 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:30,458 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524590211_0001_1_00_000000_0
2016-05-17 22:36:30,460 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:30,460 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:30,462 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524590211_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:30,464 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=8176ECEC23B44B96B4E69B168A5EE793, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:30,466 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:30,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 8176ECEC23B44B96B4E69B168A5EE793
2016-05-17 22:36:30,467 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 8176ECEC23B44B96B4E69B168A5EE793
2016-05-17 22:36:30,468 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:30,473 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D845478A286243E0895D1C279E0D1DF5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:30,474 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:30,474 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D845478A286243E0895D1C279E0D1DF5
2016-05-17 22:36:30,474 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:30,474 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D845478A286243E0895D1C279E0D1DF5
2016-05-17 22:36:30,475 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:30,476 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 8176ECEC23B44B96B4E69B168A5EE793
2016-05-17 22:36:30,476 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 8176ECEC23B44B96B4E69B168A5EE793
2016-05-17 22:36:30,474 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:30,489 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B6024A6439EE4A71BBE9884FF4997A06, ordinal: 0
2016-05-17 22:36:30,489 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B6024A6439EE4A71BBE9884FF4997A06, mem on start (mb), free: 281, total: 387, max: 672
2016-05-17 22:36:30,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:30,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:30,490 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:30,491 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:30,492 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D845478A286243E0895D1C279E0D1DF5
2016-05-17 22:36:30,492 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:30,493 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:30,493 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:30,494 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:30,494 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:30,494 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:30,495 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:30,495 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:30,495 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 8176ECEC23B44B96B4E69B168A5EE793, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:30,496 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D845478A286243E0895D1C279E0D1DF5, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:30,496 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:30,496 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 8176ECEC23B44B96B4E69B168A5EE793
2016-05-17 22:36:30,497 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:30,497 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 8176ECEC23B44B96B4E69B168A5EE793
2016-05-17 22:36:30,497 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 8176ECEC23B44B96B4E69B168A5EE793 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:30,498 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:30,498 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:30,498 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:30,499 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524590211_0001_1_00_000000_0
2016-05-17 22:36:30,500 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:30,501 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@f938429
2016-05-17 22:36:30,502 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@f938429
2016-05-17 22:36:30,503 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:30,503 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:30,502 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:30,504 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 8176ECEC23B44B96B4E69B168A5EE793
2016-05-17 22:36:30,504 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_RESULTS"], id: D845478A286243E0895D1C279E0D1DF5
2016-05-17 22:36:30,504 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_RESULTS"]
2016-05-17 22:36:30,505 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:30,505 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:30,505 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B6024A6439EE4A71BBE9884FF4997A06, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:30,508 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:30,509 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:30,509 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524590211_0001_1_00_000000_0
2016-05-17 22:36:30,510 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524590211_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:30,510 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524590211_0001_1_00_000000_0
2016-05-17 22:36:30,511 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B6024A6439EE4A71BBE9884FF4997A06, mem on close (mb), free: 280, total: 387, max: 672
2016-05-17 22:36:30 Completed running task attempt: attempt_1463524590211_0001_1_00_000000_0
2016-05-17 22:36:30,511 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524590211_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:30,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406323200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524590505, Process_Duration=5, Process_End_Time=1463524590510, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:30,513 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:30,513 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:30,513 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:30,513 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524590211_0001_00_000001
2016-05-17 22:36:30,513 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590211_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B6024A6439EE4A71BBE9884FF4997A06, taskAttemptId=attempt_1463524590211_0001_1_00_000000_0, startTime=1463524590447, finishTime=1463524590512, timeTaken=65, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406323200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524590505, Process_Duration=5, Process_End_Time=1463524590510, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:30,514 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524590211_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:30,514 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524590211_0001_1_00_000000_0
2016-05-17 22:36:30,515 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590211_0001_1][Event:TASK_FINISHED]: vertexName=B6024A6439EE4A71BBE9884FF4997A06, taskId=task_1463524590211_0001_1_00_000000, startTime=1463524590447, finishTime=1463524590515, timeTaken=68, status=SUCCEEDED, successfulAttemptID=attempt_1463524590211_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406323200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524590505, Process_Duration=5, Process_End_Time=1463524590510, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:30,516 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524590211_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:30,516 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06] : 1
2016-05-17 22:36:30,516 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06]
2016-05-17 22:36:30,517 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:30,518 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:30,519 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590211_0001_1][Event:VERTEX_FINISHED]: vertexName=B6024A6439EE4A71BBE9884FF4997A06, vertexId=vertex_1463524590211_0001_1_00, initRequestedTime=1463524590410, initedTime=1463524590429, startRequestedTime=1463524590430, startedTime=1463524590430, finishTime=1463524590516, timeTaken=86, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406323200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524590505, Process_Duration=5, Process_End_Time=1463524590510, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524590447, firstTasksToStart=[ task_1463524590211_0001_1_00_000000 ], lastTaskFinishTime=1463524590512, lastTasksToFinish=[ task_1463524590211_0001_1_00_000000 ], minTaskDuration=65, maxTaskDuration=65, avgTaskDuration=65.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524590211_0001_1_00_000000 ], longestDurationTasks=[ task_1463524590211_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:30,519 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:30,519 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524590211_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:30,519 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590211_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524590211_0001_00_000001, stoppedTime=1463524590519, exitStatus=0
2016-05-17 22:36:30,520 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524590211_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:30,520 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:30,520 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524590211_0001_1
2016-05-17 22:36:30,520 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:30,521 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:30,522 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590211_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524590211_0001_1
2016-05-17 22:36:30,522 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: D845478A286243E0895D1C279E0D1DF5 for vertex: vertex_1463524590211_0001_1_00 [B6024A6439EE4A71BBE9884FF4997A06]
2016-05-17 22:36:30,524 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524590211_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:30,524 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524590211_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:30,525 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524590211_0001_00_000001, Exiting
2016-05-17 22:36:30,525 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524590211_0001_00_000001
2016-05-17 22:36:30,525 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524590211_0001_00_000001
2016-05-17 22:36:30,526 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524590211_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:30,526 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524590211_0001_00_000001 completed successfully
2016-05-17 22:36:30,535 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524590211_0001_1, queueSize=0
2016-05-17 22:36:30,535 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:30,537 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590211_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524590211_0001_1, startTime=1463524590410, finishTime=1463524590533, timeTaken=123, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406323200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524590505, Process_Duration=5, Process_End_Time=1463524590510, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:30,537 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524590211_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:30,538 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524590211_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:30,539 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524590211_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:30,539 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524590211_0001_00_000001
2016-05-17 22:36:30 Completed Dag: dag_1463524590211_0001_1
2016-05-17 22:36:30,540 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524590211_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:30,540 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524590211_0001_1_post
2016-05-17 22:36:30,541 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:30,541 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:30,545 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/-1_UNKNOWN_RESULTS, applicationId=application_1463524590211_0001
2016-05-17 22:36:30,548 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:30,548 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:30,548 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:30,548 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:30,548 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:30,549 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:30,549 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:30,549 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:30,549 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:30,550 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:30,550 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:30,550 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:30,550 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:30,555 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4BF9BD2E29E140B78FCFD374E8FC0162/D15B4C66AAEF45A9886AA1D521D4E7C3/.tez/application_1463524590211_0001
2016-05-17 22:36:30,555 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_RESULTS/_temporary
2016-05-17 22:36:30,556 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.368
2016-05-17 22:36:30,632 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_UNKNOWN_RESULTS
2016-05-17 22:36:30,658 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:30,659 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:30,700 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 22:36:30,711 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-05-17 22:36:30,712 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:30,712 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:30,712 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:30,728 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:30,728 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_ALL
2016-05-17 22:36:30,739 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:30,739 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:30,739 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_ALL"]
2016-05-17 22:36:30,739 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:30,739 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:30,740 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:30,741 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...lyplatform/-1_UNKNOWN_ALL
2016-05-17 22:36:30,741 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:30,755 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:30,760 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:30,761 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:30,761 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:30,770 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3FC3A9D97546410E9FCC64F5FA5F604B/D0B92E480A9648D4BC088339C3B6A95C/.tez/application_1463524590760_0001 doesn't exist and is created
2016-05-17 22:36:30,788 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:30,788 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:30,789 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3FC3A9D97546410E9FCC64F5FA5F604B/D0B92E480A9648D4BC088339C3B6A95C/.tez/application_1463524590760_0001
2016-05-17 22:36:30,793 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524590760_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:30,804 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:30,805 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:30,807 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:30,807 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:30,807 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3FC3A9D97546410E9FCC64F5FA5F604B/D0B92E480A9648D4BC088339C3B6A95C/.tez/application_1463524590760_0001/localmode-log-dir
2016-05-17 22:36:30,808 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3FC3A9D97546410E9FCC64F5FA5F604B/D0B92E480A9648D4BC088339C3B6A95C/.tez/application_1463524590760_0001/localmode-log-dir/history.txt.appattempt_1463524590760_0001_000000, maxErrors=10
2016-05-17 22:36:30,808 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:30,808 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524590760_0001_000000, appSubmitTime=1463524590792, launchTime=1463524590792
2016-05-17 22:36:30,816 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:30,816 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:30,818 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:60342
2016-05-17 22:36:30,819 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:30,819 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:30,820 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:30,823 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524590760_0001_000000, startTime=1463524590823
2016-05-17 22:36:30,823 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:30,823 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:30,888 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:30,888 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:30,888 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/-1_UNKNOWN_ALL, applicationId=application_1463524590760_0001, dagName=[3FC3A9D97546410E9FCC64F5FA5F604B/D0B92E480A9648D4BC088339C3B6A95C] each/(1/1) ...lyplatform/-1_UNKNOWN_ALL
2016-05-17 22:36:30,908 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [3FC3A9D97546410E9FCC64F5FA5F604B/D0B92E480A9648D4BC088339C3B6A95C] each/(1/1) ...lyplatform/-1_UNKNOWN_ALL
2016-05-17 22:36:30,909 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524590760_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3FC3A9D97546410E9FCC64F5FA5F604B/D0B92E480A9648D4BC088339C3B6A95C/.tez/application_1463524590760_0001/localmode-log-dir/dag_1463524590760_0001_1.dot
2016-05-17 22:36:30,910 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3FC3A9D97546410E9FCC64F5FA5F604B/D0B92E480A9648D4BC088339C3B6A95C/.tez/application_1463524590760_0001/localmode-log-dir/dag_1463524590760_0001_1-tez-dag.pb.txt
2016-05-17 22:36:30,921 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524590760_0001_1
2016-05-17 22:36:30,921 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:30,921 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [3FC3A9D97546410E9FCC64F5FA5F604B/D0B92E480A9648D4BC088339C3B6A95C] each/(1/1) ...lyplatform/-1_UNKNOWN_ALL
2016-05-17 22:36:30 Running Dag: dag_1463524590760_0001_1
2016-05-17 22:36:30,934 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590760_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524590760_0001_1, submitTime=1463524590908
2016-05-17 22:36:30,934 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:30,934 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E113DECE2829403291F417589FA143E2
2016-05-17 22:36:30,935 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:30,935 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:30,936 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590760_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524590760_0001_1, initTime=1463524590934
2016-05-17 22:36:30,936 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524590760_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:30,937 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590760_0001_1][Event:DAG_STARTED]: dagID=dag_1463524590760_0001_1, startTime=1463524590937
2016-05-17 22:36:30,937 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:30,937 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524590760_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:30,937 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E113DECE2829403291F417589FA143E2 : {A8AD4B8CD7ED4B16BB2D073DD0819BF3={InputName=A8AD4B8CD7ED4B16BB2D073DD0819BF3}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:30,938 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A8AD4B8CD7ED4B16BB2D073DD0819BF3, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:30,938 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2]
2016-05-17 22:36:30,938 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2]
2016-05-17 22:36:30,938 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2]
2016-05-17 22:36:30,938 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:30,938 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2]. Starting root input initializers: 1
2016-05-17 22:36:30,940 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:30,940 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A8AD4B8CD7ED4B16BB2D073DD0819BF3 on vertex vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2]
2016-05-17 22:36:30,941 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/-1_UNKNOWN_ALL, applicationId=application_1463524590760_0001, dagName=[3FC3A9D97546410E9FCC64F5FA5F604B/D0B92E480A9648D4BC088339C3B6A95C] each/(1/1) ...lyplatform/-1_UNKNOWN_ALL
2016-05-17 22:36:30,942 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:30,942 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A8AD4B8CD7ED4B16BB2D073DD0819BF3 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:30,944 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:30,944 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:30,946 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:30,947 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A8AD4B8CD7ED4B16BB2D073DD0819BF3 on vertex vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2]
2016-05-17 22:36:30,947 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:30,947 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A8AD4B8CD7ED4B16BB2D073DD0819BF3=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:30,947 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2] parallelism set to 1
2016-05-17 22:36:30,947 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:30,948 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590760_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524590760_0001_1_00, eventCount=1
2016-05-17 22:36:30,948 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2]
2016-05-17 22:36:30,948 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=0C1E418A8CC347468FE5C526E209F882, vertexId=vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:30,948 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=0C1E418A8CC347468FE5C526E209F882, vertexId=vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2]
2016-05-17 22:36:30,951 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=0C1E418A8CC347468FE5C526E209F882, vertexName=E113DECE2829403291F417589FA143E2, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:30,952 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=0C1E418A8CC347468FE5C526E209F882, vertexId=vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2]
2016-05-17 22:36:30,954 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:30,954 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590760_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E113DECE2829403291F417589FA143E2, vertexId=vertex_1463524590760_0001_1_00, initRequestedTime=1463524590937, initedTime=1463524590953, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:30,954 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2] with distanceFromRoot: 0
2016-05-17 22:36:30,954 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:30,955 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E113DECE2829403291F417589FA143E2
2016-05-17 22:36:30,955 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2]
2016-05-17 22:36:30,955 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:30,956 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590760_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524590760_0001_1_00, startRequestedTime=1463524590955, startedTime=1463524590955
2016-05-17 22:36:30,956 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:30,957 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:30,957 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590760_0001_1][Event:TASK_STARTED]: vertexName=E113DECE2829403291F417589FA143E2, taskId=task_1463524590760_0001_1_00_000000, scheduledTime=1463524590957, launchTime=1463524590957
2016-05-17 22:36:30,958 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524590760_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:30,958 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2]
2016-05-17 22:36:30,958 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3FC3A9D97546410E9FCC64F5FA5F604B/D0B92E480A9648D4BC088339C3B6A95C] each/(1/1) ...lyplatform/-1_UNKNOWN_ALL, VertexName: E113DECE2829403291F417589FA143E2, VertexParallelism: 1, TaskAttemptID:attempt_1463524590760_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A8AD4B8CD7ED4B16BB2D073DD0819BF3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=0C1E418A8CC347468FE5C526E209F882, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:30,959 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524590760_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:30,959 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:30,960 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:30,960 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:30,961 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:30,961 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524590760_0001_00_000001
2016-05-17 22:36:30,961 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524590760_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:30,963 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524590760_0001_00_000001
2016-05-17 22:36:30,963 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:30,965 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524590760_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:30,965 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:30,965 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524590760_0001, with dag id: dag_1463524590760_0001_1
2016-05-17 22:36:30,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590760_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524590760_0001_00_000001, launchTime=1463524590965
2016-05-17 22:36:30,968 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524590760_0001_00_000001
2016-05-17 22:36:30,970 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524590760_0001_1_00_000000_0] to container: [container_1463524590760_0001_00_000001]
2016-05-17 22:36:30,971 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524590760_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:30,972 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524590760_0001_1_00_000000_0] started. Is using containerId: [container_1463524590760_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:30,972 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590760_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E113DECE2829403291F417589FA143E2, taskAttemptId=attempt_1463524590760_0001_1_00_000000_0, startTime=1463524590972, containerId=container_1463524590760_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524590760_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:30,973 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524590760_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:30,973 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:30,973 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524590760_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:30,973 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524590760_0001_00_000001 given task: attempt_1463524590760_0001_1_00_000000_0
2016-05-17 22:36:30,973 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524590760_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524590760_0001_1_00_000000_0
2016-05-17 22:36:30 Starting to run new task attempt: attempt_1463524590760_0001_1_00_000000_0
2016-05-17 22:36:30,974 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524590760_0001_1_00_000000_0
2016-05-17 22:36:30,974 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:30,974 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:30,974 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:30,975 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:30,975 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:30,975 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:30,975 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3FC3A9D97546410E9FCC64F5FA5F604B/D0B92E480A9648D4BC088339C3B6A95C] each/(1/1) ...lyplatform/-1_UNKNOWN_ALL, VertexName: E113DECE2829403291F417589FA143E2, VertexParallelism: 1, TaskAttemptID:attempt_1463524590760_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A8AD4B8CD7ED4B16BB2D073DD0819BF3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=0C1E418A8CC347468FE5C526E209F882, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:30,975 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:30,976 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524590760_0001_1_00_000000_0
2016-05-17 22:36:30,977 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:30,977 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524590760_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:30,977 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:30,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A8AD4B8CD7ED4B16BB2D073DD0819BF3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:30,979 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:30,979 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=0C1E418A8CC347468FE5C526E209F882, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:30,979 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:30,980 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 0C1E418A8CC347468FE5C526E209F882
2016-05-17 22:36:30,980 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:30,980 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 0C1E418A8CC347468FE5C526E209F882
2016-05-17 22:36:30,980 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:30,981 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A8AD4B8CD7ED4B16BB2D073DD0819BF3
2016-05-17 22:36:30,981 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:30,981 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A8AD4B8CD7ED4B16BB2D073DD0819BF3
2016-05-17 22:36:30,987 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:30,987 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A8AD4B8CD7ED4B16BB2D073DD0819BF3
2016-05-17 22:36:30,987 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A8AD4B8CD7ED4B16BB2D073DD0819BF3
2016-05-17 22:36:30,994 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E113DECE2829403291F417589FA143E2, ordinal: 0
2016-05-17 22:36:30,994 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E113DECE2829403291F417589FA143E2, mem on start (mb), free: 201, total: 387, max: 672
2016-05-17 22:36:30,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:30,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:30,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:30,997 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:30,997 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 0C1E418A8CC347468FE5C526E209F882
2016-05-17 22:36:30,997 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:30,997 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:30,998 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:30,998 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:30,998 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:30,998 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:30,999 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:30,999 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:30,999 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 0C1E418A8CC347468FE5C526E209F882, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:30,999 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A8AD4B8CD7ED4B16BB2D073DD0819BF3, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:31,000 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:31,000 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A8AD4B8CD7ED4B16BB2D073DD0819BF3
2016-05-17 22:36:31,000 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:31,000 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A8AD4B8CD7ED4B16BB2D073DD0819BF3
2016-05-17 22:36:31,000 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A8AD4B8CD7ED4B16BB2D073DD0819BF3 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:31,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:31,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:31,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:31,002 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524590760_0001_1_00_000000_0
2016-05-17 22:36:31,002 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:31,003 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@36699775
2016-05-17 22:36:31,004 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@36699775
2016-05-17 22:36:31,004 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:31,004 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:31,004 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:31,005 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: A8AD4B8CD7ED4B16BB2D073DD0819BF3
2016-05-17 22:36:31,005 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_ALL"], id: 0C1E418A8CC347468FE5C526E209F882
2016-05-17 22:36:31,005 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_ALL"]
2016-05-17 22:36:31,005 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:31,006 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:31,006 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E113DECE2829403291F417589FA143E2, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:31,008 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:31,009 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:31,009 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524590760_0001_1_00_000000_0
2016-05-17 22:36:31,009 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524590760_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:31,010 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524590760_0001_1_00_000000_0
2016-05-17 22:36:31,010 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E113DECE2829403291F417589FA143E2, mem on close (mb), free: 199, total: 387, max: 672
2016-05-17 22:36:31 Completed running task attempt: attempt_1463524590760_0001_1_00_000000_0
2016-05-17 22:36:31,011 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524590760_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:31,012 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:31,013 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590760_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E113DECE2829403291F417589FA143E2, taskAttemptId=attempt_1463524590760_0001_1_00_000000_0, startTime=1463524590972, finishTime=1463524591012, timeTaken=40, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406323200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524591006, Process_Duration=4, Process_End_Time=1463524591010, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:31,013 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524590760_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:31,013 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:31,013 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524590760_0001_1_00_000000_0
2016-05-17 22:36:31,014 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:31,014 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590760_0001_1][Event:TASK_FINISHED]: vertexName=E113DECE2829403291F417589FA143E2, taskId=task_1463524590760_0001_1_00_000000, startTime=1463524590972, finishTime=1463524591014, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1463524590760_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406323200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524591006, Process_Duration=4, Process_End_Time=1463524591010, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:31,014 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524590760_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:31,015 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2] : 1
2016-05-17 22:36:31,015 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2]
2016-05-17 22:36:31,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406323200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524591006, Process_Duration=4, Process_End_Time=1463524591010, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:31,016 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:31,016 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:31,016 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:31,017 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524590760_0001_00_000001
2016-05-17 22:36:31,019 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590760_0001_1][Event:VERTEX_FINISHED]: vertexName=E113DECE2829403291F417589FA143E2, vertexId=vertex_1463524590760_0001_1_00, initRequestedTime=1463524590937, initedTime=1463524590953, startRequestedTime=1463524590955, startedTime=1463524590955, finishTime=1463524591015, timeTaken=60, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406323200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524591006, Process_Duration=4, Process_End_Time=1463524591010, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524590972, firstTasksToStart=[ task_1463524590760_0001_1_00_000000 ], lastTaskFinishTime=1463524591012, lastTasksToFinish=[ task_1463524590760_0001_1_00_000000 ], minTaskDuration=40, maxTaskDuration=40, avgTaskDuration=40.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524590760_0001_1_00_000000 ], longestDurationTasks=[ task_1463524590760_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:31,020 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:31,020 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524590760_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:31,020 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590760_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524590760_0001_00_000001, stoppedTime=1463524591020, exitStatus=0
2016-05-17 22:36:31,020 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:31,020 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524590760_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:31,020 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:31,020 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524590760_0001_1
2016-05-17 22:36:31,021 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590760_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524590760_0001_1
2016-05-17 22:36:31,021 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 0C1E418A8CC347468FE5C526E209F882 for vertex: vertex_1463524590760_0001_1_00 [E113DECE2829403291F417589FA143E2]
2016-05-17 22:36:31,027 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524590760_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:31,027 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524590760_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:31,028 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524590760_0001_00_000001, Exiting
2016-05-17 22:36:31,028 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524590760_0001_00_000001
2016-05-17 22:36:31,028 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524590760_0001_00_000001
2016-05-17 22:36:31,030 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524590760_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:31,031 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524590760_0001_00_000001 completed successfully
2016-05-17 22:36:31,032 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524590760_0001_1, queueSize=0
2016-05-17 22:36:31,032 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:31,032 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524590760_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524590760_0001_1, startTime=1463524590937, finishTime=1463524591030, timeTaken=93, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406323200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524591006, Process_Duration=4, Process_End_Time=1463524591010, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:31,033 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524590760_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:31,033 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524590760_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:31,033 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524590760_0001_00_000001
2016-05-17 22:36:31,033 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524590760_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:31 Completed Dag: dag_1463524590760_0001_1
2016-05-17 22:36:31,034 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524590760_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:31,034 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524590760_0001_1_post
2016-05-17 22:36:31,034 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:31,034 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:31,040 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/-1_UNKNOWN_ALL, applicationId=application_1463524590760_0001
2016-05-17 22:36:31,042 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:31,042 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:31,042 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:31,042 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:31,042 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:31,043 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:31,043 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:31,044 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:31,045 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:31,045 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:31,045 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:31,046 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:31,046 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:31,050 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3FC3A9D97546410E9FCC64F5FA5F604B/D0B92E480A9648D4BC088339C3B6A95C/.tez/application_1463524590760_0001
2016-05-17 22:36:31,050 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_ALL/_temporary
2016-05-17 22:36:31,051 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.311
2016-05-17 22:36:31,134 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_UNKNOWN_ALL
2016-05-17 22:36:31,160 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:31,160 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:31,203 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 22:36:31,211 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-05-17 22:36:31,211 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:31,211 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:31,212 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:31,237 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:31,238 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_REPLACE
2016-05-17 22:36:31,249 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:31,250 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:31,250 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_REPLACE"]
2016-05-17 22:36:31,250 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:31,250 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:31,250 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:31,251 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...atform/-1_UNKNOWN_REPLACE
2016-05-17 22:36:31,251 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:31,265 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:31,269 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:31,270 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:31,270 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:31,281 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CFCEE0926A8F4FE1A1ABAA34B9240492/CF50378B89914A52B3A998BEDBB8630C/.tez/application_1463524591270_0001 doesn't exist and is created
2016-05-17 22:36:31,299 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:31,300 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:31,300 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CFCEE0926A8F4FE1A1ABAA34B9240492/CF50378B89914A52B3A998BEDBB8630C/.tez/application_1463524591270_0001
2016-05-17 22:36:31,304 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524591270_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:31,326 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:31,327 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:31,330 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:31,330 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:31,332 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CFCEE0926A8F4FE1A1ABAA34B9240492/CF50378B89914A52B3A998BEDBB8630C/.tez/application_1463524591270_0001/localmode-log-dir
2016-05-17 22:36:31,332 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CFCEE0926A8F4FE1A1ABAA34B9240492/CF50378B89914A52B3A998BEDBB8630C/.tez/application_1463524591270_0001/localmode-log-dir/history.txt.appattempt_1463524591270_0001_000000, maxErrors=10
2016-05-17 22:36:31,332 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:31,334 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524591270_0001_000000, appSubmitTime=1463524591303, launchTime=1463524591304
2016-05-17 22:36:31,337 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:31,338 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:31,342 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:31,342 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:31,342 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:50687
2016-05-17 22:36:31,345 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:31,348 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524591270_0001_000000, startTime=1463524591348
2016-05-17 22:36:31,348 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:31,348 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:31,400 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:31,400 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:31,400 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/-1_UNKNOWN_REPLACE, applicationId=application_1463524591270_0001, dagName=[CFCEE0926A8F4FE1A1ABAA34B9240492/CF50378B89914A52B3A998BEDBB8630C] each/(1/1) ...atform/-1_UNKNOWN_REPLACE
2016-05-17 22:36:31,433 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [CFCEE0926A8F4FE1A1ABAA34B9240492/CF50378B89914A52B3A998BEDBB8630C] each/(1/1) ...atform/-1_UNKNOWN_REPLACE
2016-05-17 22:36:31,435 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524591270_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CFCEE0926A8F4FE1A1ABAA34B9240492/CF50378B89914A52B3A998BEDBB8630C/.tez/application_1463524591270_0001/localmode-log-dir/dag_1463524591270_0001_1.dot
2016-05-17 22:36:31,435 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CFCEE0926A8F4FE1A1ABAA34B9240492/CF50378B89914A52B3A998BEDBB8630C/.tez/application_1463524591270_0001/localmode-log-dir/dag_1463524591270_0001_1-tez-dag.pb.txt
2016-05-17 22:36:31,441 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524591270_0001_1
2016-05-17 22:36:31,442 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:31,442 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [CFCEE0926A8F4FE1A1ABAA34B9240492/CF50378B89914A52B3A998BEDBB8630C] each/(1/1) ...atform/-1_UNKNOWN_REPLACE
2016-05-17 22:36:31 Running Dag: dag_1463524591270_0001_1
2016-05-17 22:36:31,455 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591270_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524591270_0001_1, submitTime=1463524591433
2016-05-17 22:36:31,455 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:31,456 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C473AA5866E74F65B95233095BF685E5
2016-05-17 22:36:31,456 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:31,456 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:31,456 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:31,457 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591270_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524591270_0001_1, initTime=1463524591455
2016-05-17 22:36:31,457 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524591270_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:31,457 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:31,458 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591270_0001_1][Event:DAG_STARTED]: dagID=dag_1463524591270_0001_1, startTime=1463524591458
2016-05-17 22:36:31,458 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:31,459 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524591270_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:31,459 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:31,459 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C473AA5866E74F65B95233095BF685E5 : {2527A4F2101D47C59F1E2FDF8B319921={InputName=2527A4F2101D47C59F1E2FDF8B319921}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:31,459 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 2527A4F2101D47C59F1E2FDF8B319921, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:31,459 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5]
2016-05-17 22:36:31,459 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5]
2016-05-17 22:36:31,459 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5]
2016-05-17 22:36:31,460 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/-1_UNKNOWN_REPLACE, applicationId=application_1463524591270_0001, dagName=[CFCEE0926A8F4FE1A1ABAA34B9240492/CF50378B89914A52B3A998BEDBB8630C] each/(1/1) ...atform/-1_UNKNOWN_REPLACE
2016-05-17 22:36:31,460 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5]. Starting root input initializers: 1
2016-05-17 22:36:31,460 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:31,461 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 2527A4F2101D47C59F1E2FDF8B319921 on vertex vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5]
2016-05-17 22:36:31,464 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:31,464 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 2527A4F2101D47C59F1E2FDF8B319921 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:31,466 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:31,467 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:31,469 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:31,470 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 2527A4F2101D47C59F1E2FDF8B319921 on vertex vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5]
2016-05-17 22:36:31,470 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:31,470 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {2527A4F2101D47C59F1E2FDF8B319921=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:31,470 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5] parallelism set to 1
2016-05-17 22:36:31,471 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:31,471 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591270_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524591270_0001_1_00, eventCount=1
2016-05-17 22:36:31,471 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5]
2016-05-17 22:36:31,471 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=F139C77FA376451BAB9CC0A78D655491, vertexId=vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:31,471 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=F139C77FA376451BAB9CC0A78D655491, vertexId=vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5]
2016-05-17 22:36:31,473 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=F139C77FA376451BAB9CC0A78D655491, vertexName=C473AA5866E74F65B95233095BF685E5, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:31,473 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=F139C77FA376451BAB9CC0A78D655491, vertexId=vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5]
2016-05-17 22:36:31,474 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:31,474 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591270_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C473AA5866E74F65B95233095BF685E5, vertexId=vertex_1463524591270_0001_1_00, initRequestedTime=1463524591459, initedTime=1463524591474, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:31,475 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5] with distanceFromRoot: 0
2016-05-17 22:36:31,475 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:31,475 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C473AA5866E74F65B95233095BF685E5
2016-05-17 22:36:31,475 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5]
2016-05-17 22:36:31,475 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:31,476 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591270_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524591270_0001_1_00, startRequestedTime=1463524591475, startedTime=1463524591475
2016-05-17 22:36:31,476 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:31,476 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:31,477 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591270_0001_1][Event:TASK_STARTED]: vertexName=C473AA5866E74F65B95233095BF685E5, taskId=task_1463524591270_0001_1_00_000000, scheduledTime=1463524591476, launchTime=1463524591476
2016-05-17 22:36:31,477 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524591270_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:31,477 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5]
2016-05-17 22:36:31,477 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [CFCEE0926A8F4FE1A1ABAA34B9240492/CF50378B89914A52B3A998BEDBB8630C] each/(1/1) ...atform/-1_UNKNOWN_REPLACE, VertexName: C473AA5866E74F65B95233095BF685E5, VertexParallelism: 1, TaskAttemptID:attempt_1463524591270_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2527A4F2101D47C59F1E2FDF8B319921, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F139C77FA376451BAB9CC0A78D655491, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:31,478 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524591270_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:31,478 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:31,478 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:31,479 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:31,479 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:31,480 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524591270_0001_00_000001
2016-05-17 22:36:31,480 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524591270_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:31,482 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524591270_0001_00_000001
2016-05-17 22:36:31,482 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:31,482 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524591270_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:31,483 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:31,483 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591270_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524591270_0001_00_000001, launchTime=1463524591483
2016-05-17 22:36:31,484 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524591270_0001_00_000001
2016-05-17 22:36:31,485 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524591270_0001_1_00_000000_0] to container: [container_1463524591270_0001_00_000001]
2016-05-17 22:36:31,485 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524591270_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:31,485 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524591270_0001_00_000001 given task: attempt_1463524591270_0001_1_00_000000_0
2016-05-17 22:36:31,486 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524591270_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524591270_0001_1_00_000000_0
2016-05-17 22:36:31,486 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524591270_0001_1_00_000000_0] started. Is using containerId: [container_1463524591270_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:31,486 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524591270_0001, with dag id: dag_1463524591270_0001_1
2016-05-17 22:36:31,486 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591270_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C473AA5866E74F65B95233095BF685E5, taskAttemptId=attempt_1463524591270_0001_1_00_000000_0, startTime=1463524591485, containerId=container_1463524591270_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524591270_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:31,487 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:31,487 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524591270_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:31 Starting to run new task attempt: attempt_1463524591270_0001_1_00_000000_0
2016-05-17 22:36:31,488 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524591270_0001_1_00_000000_0
2016-05-17 22:36:31,488 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:31,488 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:31,489 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:31,488 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524591270_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:31,490 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:31,490 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:31,491 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:31,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [CFCEE0926A8F4FE1A1ABAA34B9240492/CF50378B89914A52B3A998BEDBB8630C] each/(1/1) ...atform/-1_UNKNOWN_REPLACE, VertexName: C473AA5866E74F65B95233095BF685E5, VertexParallelism: 1, TaskAttemptID:attempt_1463524591270_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2527A4F2101D47C59F1E2FDF8B319921, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F139C77FA376451BAB9CC0A78D655491, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:31,491 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:31,492 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524591270_0001_1_00_000000_0
2016-05-17 22:36:31,492 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:31,492 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:31,493 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524591270_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:31,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=2527A4F2101D47C59F1E2FDF8B319921, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:31,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:31,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:31,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 2527A4F2101D47C59F1E2FDF8B319921
2016-05-17 22:36:31,496 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 2527A4F2101D47C59F1E2FDF8B319921
2016-05-17 22:36:31,497 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F139C77FA376451BAB9CC0A78D655491, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:31,498 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:31,498 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:31,498 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F139C77FA376451BAB9CC0A78D655491
2016-05-17 22:36:31,498 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:31,499 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: F139C77FA376451BAB9CC0A78D655491
2016-05-17 22:36:31,500 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:31,500 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 2527A4F2101D47C59F1E2FDF8B319921
2016-05-17 22:36:31,500 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 2527A4F2101D47C59F1E2FDF8B319921
2016-05-17 22:36:31,507 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C473AA5866E74F65B95233095BF685E5, ordinal: 0
2016-05-17 22:36:31,507 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C473AA5866E74F65B95233095BF685E5, mem on start (mb), free: 336, total: 388, max: 672
2016-05-17 22:36:31,507 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:31,508 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:31,508 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:31,510 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:31,510 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F139C77FA376451BAB9CC0A78D655491
2016-05-17 22:36:31,510 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:31,510 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:31,511 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:31,511 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:31,511 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:31,511 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:31,511 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:31,511 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:31,511 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 2527A4F2101D47C59F1E2FDF8B319921, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:31,512 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F139C77FA376451BAB9CC0A78D655491, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:31,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:31,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 2527A4F2101D47C59F1E2FDF8B319921 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:31,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:31,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:31,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 2527A4F2101D47C59F1E2FDF8B319921
2016-05-17 22:36:31,512 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:31,513 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 2527A4F2101D47C59F1E2FDF8B319921
2016-05-17 22:36:31,513 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:31,513 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524591270_0001_1_00_000000_0
2016-05-17 22:36:31,513 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:31,514 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4fabbfd2
2016-05-17 22:36:31,515 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4fabbfd2
2016-05-17 22:36:31,515 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:31,515 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:31,516 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:31,517 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 2527A4F2101D47C59F1E2FDF8B319921
2016-05-17 22:36:31,517 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_REPLACE"], id: F139C77FA376451BAB9CC0A78D655491
2016-05-17 22:36:31,517 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_REPLACE"]
2016-05-17 22:36:31,517 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:31,518 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:31,518 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C473AA5866E74F65B95233095BF685E5, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:31,519 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:31,525 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:31,529 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:31,529 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:31,530 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524591270_0001_1_00_000000_0
2016-05-17 22:36:31,530 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524591270_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:31,531 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524591270_0001_1_00_000000_0
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:31,539 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524591518, Process_Duration=13, Process_End_Time=1463524591531, Read_Duration=1, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:36:31,539 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C473AA5866E74F65B95233095BF685E5, mem on close (mb), free: 334, total: 388, max: 672
2016-05-17 22:36:31,539 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=C473AA5866E74F65B95233095BF685E5, index=0
2016-05-17 22:36:31,540 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=C473AA5866E74F65B95233095BF685E5, sourceVertex=2527A4F2101D47C59F1E2FDF8B319921
2016-05-17 22:36:31,540 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=C473AA5866E74F65B95233095BF685E5, sourceVertex=F139C77FA376451BAB9CC0A78D655491
2016-05-17 22:36:31,541 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524591270_0001_00_000001
2016-05-17 22:36:31,541 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524591270_0001_00_000001
2016-05-17 22:36:31,541 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:31,541 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591270_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C473AA5866E74F65B95233095BF685E5, taskAttemptId=attempt_1463524591270_0001_1_00_000000_0, startTime=1463524591485, finishTime=1463524591539, timeTaken=54, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524591518, Process_Duration=13, Process_End_Time=1463524591531, Read_Duration=1, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:36:31,542 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524591270_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:36:31,543 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524591270_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:36:31,543 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524591270_0001_00_000001 completed but with errors
2016-05-17 22:36:31,543 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524591270_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:36:31,544 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591270_0001_1][Event:TASK_FINISHED]: vertexName=C473AA5866E74F65B95233095BF685E5, taskId=task_1463524591270_0001_1_00_000000, startTime=1463524591485, finishTime=1463524591543, timeTaken=58, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
], counters=Counters: 0
2016-05-17 22:36:31,544 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:31,545 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524591270_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:36:31,546 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:31,548 INFO container.AMContainerImpl (AMContainerImpl.java:transition(686)) - Container container_1463524591270_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:36:31,549 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:31,549 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591270_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524591270_0001_00_000001, stoppedTime=1463524591548, exitStatus=1
2016-05-17 22:36:31,549 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524591270_0001_00_000001 transitioned from RUNNING to COMPLETED via event C_COMPLETED
2016-05-17 22:36:31,550 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5] : 1
2016-05-17 22:36:31,550 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5] because task failed: task_1463524591270_0001_1_00_000000
2016-05-17 22:36:31,550 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:36:31,551 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:36:31,551 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5]
2016-05-17 22:36:31,552 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=F139C77FA376451BAB9CC0A78D655491, vertexId=vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5]
2016-05-17 22:36:31,557 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:31,558 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591270_0001_1][Event:VERTEX_FINISHED]: vertexName=C473AA5866E74F65B95233095BF685E5, vertexId=vertex_1463524591270_0001_1_00, initRequestedTime=1463524591459, initedTime=1463524591474, startRequestedTime=1463524591475, startedTime=1463524591475, finishTime=1463524591556, timeTaken=81, status=FAILED, diagnostics=Task failed, taskId=task_1463524591270_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524591485, firstTasksToStart=[ task_1463524591270_0001_1_00_000000 ], lastTaskFinishTime=1463524591539, lastTasksToFinish=[ task_1463524591270_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:36:31,558 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:36:31,559 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524591270_0001_1_00_000000_0 failed: true container: container_1463524591270_0001_00_000001 numFailedTAs: 0
2016-05-17 22:36:31,559 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524591270_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:36:31,562 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:36:31,562 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:36:31,562 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524591270_0001_1
2016-05-17 22:36:31,562 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: F139C77FA376451BAB9CC0A78D655491 for vertex: vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5]
2016-05-17 22:36:31,564 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524591270_0001_1, queueSize=0
2016-05-17 22:36:31,564 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:31,565 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591270_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524591270_0001_1, startTime=1463524591458, finishTime=1463524591563, timeTaken=105, status=FAILED, diagnostics=Vertex failed, vertexName=C473AA5866E74F65B95233095BF685E5, vertexId=vertex_1463524591270_0001_1_00, diagnostics=[Task failed, taskId=task_1463524591270_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524591270_0001_1_00 [C473AA5866E74F65B95233095BF685E5] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:36:31,565 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524591270_0001_1 finished with state: FAILED
2016-05-17 22:36:31,565 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524591270_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:31 Completed Dag: dag_1463524591270_0001_1
2016-05-17 22:36:31,567 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524591270_0001_1, dagState=FAILED
2016-05-17 22:36:31,567 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524591270_0001_1_post
2016-05-17 22:36:31,567 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:31,567 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:31,570 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/-1_UNKNOWN_REPLACE, applicationId=application_1463524591270_0001
2016-05-17 22:36:31,573 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:31,573 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:31,573 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:31,573 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:31,573 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:31,574 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:31,576 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:31,576 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:31,576 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:31,577 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:31,577 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:31,577 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:31,577 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:31,582 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CFCEE0926A8F4FE1A1ABAA34B9240492/CF50378B89914A52B3A998BEDBB8630C/.tez/application_1463524591270_0001
2016-05-17 22:36:31,582 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping all jobs
2016-05-17 22:36:31,582 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping: (1/1) ...atform/-1_UNKNOWN_REPLACE
2016-05-17 22:36:31,583 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopped all jobs
2016-05-17 22:36:31,584 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_REPLACE/_temporary
2016-05-17 22:36:31,584 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.334
2016-05-17 22:36:31,584 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_UNKNOWN_REPLACE
2016-05-17 22:36:31,616 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:31,616 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:31,651 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 22:36:31,662 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-05-17 22:36:31,662 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:31,662 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:31,663 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:31,677 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:31,678 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_SWAP
2016-05-17 22:36:31,688 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:31,689 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:31,689 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_SWAP"]
2016-05-17 22:36:31,690 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:31,690 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:31,690 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:31,692 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...yplatform/-1_UNKNOWN_SWAP
2016-05-17 22:36:31,692 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:31,704 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:31,710 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:31,711 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:31,711 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:31,721 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/76B73064343B42D5AE3288D21D90EC7D/D8DC6232DCD2446481F80D6DA83465CF/.tez/application_1463524591710_0001 doesn't exist and is created
2016-05-17 22:36:31,741 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:31,742 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:31,742 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/76B73064343B42D5AE3288D21D90EC7D/D8DC6232DCD2446481F80D6DA83465CF/.tez/application_1463524591710_0001
2016-05-17 22:36:31,748 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524591710_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:31,764 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:31,766 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:31,768 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:31,768 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:31,768 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/76B73064343B42D5AE3288D21D90EC7D/D8DC6232DCD2446481F80D6DA83465CF/.tez/application_1463524591710_0001/localmode-log-dir
2016-05-17 22:36:31,769 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/76B73064343B42D5AE3288D21D90EC7D/D8DC6232DCD2446481F80D6DA83465CF/.tez/application_1463524591710_0001/localmode-log-dir/history.txt.appattempt_1463524591710_0001_000000, maxErrors=10
2016-05-17 22:36:31,769 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:31,770 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524591710_0001_000000, appSubmitTime=1463524591747, launchTime=1463524591747
2016-05-17 22:36:31,774 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:31,775 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:31,777 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:31,777 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:31,780 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:59015
2016-05-17 22:36:31,782 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:31,784 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524591710_0001_000000, startTime=1463524591783
2016-05-17 22:36:31,784 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:31,784 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:31,842 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:31,842 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:31,842 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/-1_UNKNOWN_SWAP, applicationId=application_1463524591710_0001, dagName=[76B73064343B42D5AE3288D21D90EC7D/D8DC6232DCD2446481F80D6DA83465CF] each/(1/1) ...yplatform/-1_UNKNOWN_SWAP
2016-05-17 22:36:31,860 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [76B73064343B42D5AE3288D21D90EC7D/D8DC6232DCD2446481F80D6DA83465CF] each/(1/1) ...yplatform/-1_UNKNOWN_SWAP
2016-05-17 22:36:31,862 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524591710_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/76B73064343B42D5AE3288D21D90EC7D/D8DC6232DCD2446481F80D6DA83465CF/.tez/application_1463524591710_0001/localmode-log-dir/dag_1463524591710_0001_1.dot
2016-05-17 22:36:31,862 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/76B73064343B42D5AE3288D21D90EC7D/D8DC6232DCD2446481F80D6DA83465CF/.tez/application_1463524591710_0001/localmode-log-dir/dag_1463524591710_0001_1-tez-dag.pb.txt
2016-05-17 22:36:31,869 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524591710_0001_1
2016-05-17 22:36:31,869 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:31,869 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [76B73064343B42D5AE3288D21D90EC7D/D8DC6232DCD2446481F80D6DA83465CF] each/(1/1) ...yplatform/-1_UNKNOWN_SWAP
2016-05-17 22:36:31 Running Dag: dag_1463524591710_0001_1
2016-05-17 22:36:31,882 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591710_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524591710_0001_1, submitTime=1463524591860
2016-05-17 22:36:31,882 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:31,882 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A8B668D33A87492F91F41094C361D602
2016-05-17 22:36:31,883 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:31,883 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591710_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524591710_0001_1, initTime=1463524591882
2016-05-17 22:36:31,883 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:31,883 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524591710_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:31,884 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591710_0001_1][Event:DAG_STARTED]: dagID=dag_1463524591710_0001_1, startTime=1463524591884
2016-05-17 22:36:31,884 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:31,885 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:31,885 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/-1_UNKNOWN_SWAP, applicationId=application_1463524591710_0001, dagName=[76B73064343B42D5AE3288D21D90EC7D/D8DC6232DCD2446481F80D6DA83465CF] each/(1/1) ...yplatform/-1_UNKNOWN_SWAP
2016-05-17 22:36:31,885 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524591710_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:31,885 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A8B668D33A87492F91F41094C361D602 : {62F3CF69ADB34C609210E04C626F31A5={InputName=62F3CF69ADB34C609210E04C626F31A5}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:31,886 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 62F3CF69ADB34C609210E04C626F31A5, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:31,886 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602]
2016-05-17 22:36:31,886 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602]
2016-05-17 22:36:31,886 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602]
2016-05-17 22:36:31,886 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602]. Starting root input initializers: 1
2016-05-17 22:36:31,887 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:31,887 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 62F3CF69ADB34C609210E04C626F31A5 on vertex vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602]
2016-05-17 22:36:31,889 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:31,889 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 62F3CF69ADB34C609210E04C626F31A5 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:31,890 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:31,890 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:31,899 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524591710_0001, with dag id: dag_1463524591710_0001_1
2016-05-17 22:36:31,900 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:31,901 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 62F3CF69ADB34C609210E04C626F31A5 on vertex vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602]
2016-05-17 22:36:31,902 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:31,902 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {62F3CF69ADB34C609210E04C626F31A5=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:31,902 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602] parallelism set to 1
2016-05-17 22:36:31,903 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:31,903 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591710_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524591710_0001_1_00, eventCount=1
2016-05-17 22:36:31,904 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602]
2016-05-17 22:36:31,904 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=7B95D1C6E8C34C6082C1BAC0D65FDE65, vertexId=vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:31,904 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=7B95D1C6E8C34C6082C1BAC0D65FDE65, vertexId=vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602]
2016-05-17 22:36:31,905 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=7B95D1C6E8C34C6082C1BAC0D65FDE65, vertexName=A8B668D33A87492F91F41094C361D602, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:31,906 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=7B95D1C6E8C34C6082C1BAC0D65FDE65, vertexId=vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602]
2016-05-17 22:36:31,907 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:31,908 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591710_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A8B668D33A87492F91F41094C361D602, vertexId=vertex_1463524591710_0001_1_00, initRequestedTime=1463524591885, initedTime=1463524591907, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:31,908 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602] with distanceFromRoot: 0
2016-05-17 22:36:31,908 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:31,908 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A8B668D33A87492F91F41094C361D602
2016-05-17 22:36:31,908 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602]
2016-05-17 22:36:31,909 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:31,909 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591710_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524591710_0001_1_00, startRequestedTime=1463524591908, startedTime=1463524591908
2016-05-17 22:36:31,909 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:31,910 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:31,910 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591710_0001_1][Event:TASK_STARTED]: vertexName=A8B668D33A87492F91F41094C361D602, taskId=task_1463524591710_0001_1_00_000000, scheduledTime=1463524591910, launchTime=1463524591910
2016-05-17 22:36:31,911 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524591710_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:31,911 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602]
2016-05-17 22:36:31,911 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [76B73064343B42D5AE3288D21D90EC7D/D8DC6232DCD2446481F80D6DA83465CF] each/(1/1) ...yplatform/-1_UNKNOWN_SWAP, VertexName: A8B668D33A87492F91F41094C361D602, VertexParallelism: 1, TaskAttemptID:attempt_1463524591710_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=62F3CF69ADB34C609210E04C626F31A5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=7B95D1C6E8C34C6082C1BAC0D65FDE65, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:31,912 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524591710_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:31,912 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:31,913 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:31,914 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:31,914 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:31,915 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524591710_0001_00_000001
2016-05-17 22:36:31,915 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524591710_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:31,917 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524591710_0001_00_000001
2016-05-17 22:36:31,917 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:31,918 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:31,918 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524591710_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:31,918 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591710_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524591710_0001_00_000001, launchTime=1463524591917
2016-05-17 22:36:31,918 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524591710_0001_00_000001
2016-05-17 22:36:31,919 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524591710_0001_1_00_000000_0] to container: [container_1463524591710_0001_00_000001]
2016-05-17 22:36:31,919 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524591710_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:31,919 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524591710_0001_00_000001 given task: attempt_1463524591710_0001_1_00_000000_0
2016-05-17 22:36:31,920 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524591710_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524591710_0001_1_00_000000_0
2016-05-17 22:36:31,920 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524591710_0001_1_00_000000_0] started. Is using containerId: [container_1463524591710_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:31,920 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:31,920 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591710_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A8B668D33A87492F91F41094C361D602, taskAttemptId=attempt_1463524591710_0001_1_00_000000_0, startTime=1463524591920, containerId=container_1463524591710_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524591710_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:31,921 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524591710_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:31,921 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524591710_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:31 Starting to run new task attempt: attempt_1463524591710_0001_1_00_000000_0
2016-05-17 22:36:31,922 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524591710_0001_1_00_000000_0
2016-05-17 22:36:31,922 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:31,922 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:31,922 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:31,923 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:31,923 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:31,923 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:31,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [76B73064343B42D5AE3288D21D90EC7D/D8DC6232DCD2446481F80D6DA83465CF] each/(1/1) ...yplatform/-1_UNKNOWN_SWAP, VertexName: A8B668D33A87492F91F41094C361D602, VertexParallelism: 1, TaskAttemptID:attempt_1463524591710_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=62F3CF69ADB34C609210E04C626F31A5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=7B95D1C6E8C34C6082C1BAC0D65FDE65, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:31,924 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:31,925 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524591710_0001_1_00_000000_0
2016-05-17 22:36:31,925 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:31,926 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:31,926 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524591710_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:31,927 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=62F3CF69ADB34C609210E04C626F31A5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:31,927 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=7B95D1C6E8C34C6082C1BAC0D65FDE65, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:31,927 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:31,928 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 62F3CF69ADB34C609210E04C626F31A5
2016-05-17 22:36:31,928 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 62F3CF69ADB34C609210E04C626F31A5
2016-05-17 22:36:31,927 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:31,928 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:31,930 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 7B95D1C6E8C34C6082C1BAC0D65FDE65
2016-05-17 22:36:31,930 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:31,930 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:31,931 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 62F3CF69ADB34C609210E04C626F31A5
2016-05-17 22:36:31,931 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 62F3CF69ADB34C609210E04C626F31A5
2016-05-17 22:36:31,931 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:31,930 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 7B95D1C6E8C34C6082C1BAC0D65FDE65
2016-05-17 22:36:31,943 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A8B668D33A87492F91F41094C361D602, ordinal: 0
2016-05-17 22:36:31,943 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A8B668D33A87492F91F41094C361D602, mem on start (mb), free: 271, total: 388, max: 672
2016-05-17 22:36:31,943 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:31,943 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:31,943 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:31,947 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:31,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 7B95D1C6E8C34C6082C1BAC0D65FDE65
2016-05-17 22:36:31,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:31,947 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:31,948 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:31,948 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:31,948 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:31,948 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:31,948 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:31,949 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:31,950 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 62F3CF69ADB34C609210E04C626F31A5, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:31,950 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 7B95D1C6E8C34C6082C1BAC0D65FDE65, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:31,951 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:31,951 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 62F3CF69ADB34C609210E04C626F31A5 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:31,951 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 62F3CF69ADB34C609210E04C626F31A5
2016-05-17 22:36:31,951 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:31,951 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 62F3CF69ADB34C609210E04C626F31A5
2016-05-17 22:36:31,951 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:31,952 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:31,952 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:31,952 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524591710_0001_1_00_000000_0
2016-05-17 22:36:31,952 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:31,953 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5f0d52a0
2016-05-17 22:36:31,954 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5f0d52a0
2016-05-17 22:36:31,954 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:31,954 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:31,956 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:31,957 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 62F3CF69ADB34C609210E04C626F31A5
2016-05-17 22:36:31,957 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_SWAP"], id: 7B95D1C6E8C34C6082C1BAC0D65FDE65
2016-05-17 22:36:31,957 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_SWAP"]
2016-05-17 22:36:31,958 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:31,958 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:31,958 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A8B668D33A87492F91F41094C361D602, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:31,961 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:31,962 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:31,963 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524591710_0001_1_00_000000_0
2016-05-17 22:36:31,963 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524591710_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:31,964 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524591710_0001_1_00_000000_0
2016-05-17 22:36:31,964 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A8B668D33A87492F91F41094C361D602, mem on close (mb), free: 269, total: 388, max: 672
2016-05-17 22:36:31 Completed running task attempt: attempt_1463524591710_0001_1_00_000000_0
2016-05-17 22:36:31,965 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524591710_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:31,966 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524591958, Process_Duration=6, Process_End_Time=1463524591964, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:31,966 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:31,967 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591710_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A8B668D33A87492F91F41094C361D602, taskAttemptId=attempt_1463524591710_0001_1_00_000000_0, startTime=1463524591920, finishTime=1463524591966, timeTaken=46, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524591958, Process_Duration=6, Process_End_Time=1463524591964, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:31,967 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:31,967 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:31,968 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524591710_0001_00_000001
2016-05-17 22:36:31,967 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524591710_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:31,968 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:31,968 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524591710_0001_1_00_000000_0
2016-05-17 22:36:31,969 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:31,969 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591710_0001_1][Event:TASK_FINISHED]: vertexName=A8B668D33A87492F91F41094C361D602, taskId=task_1463524591710_0001_1_00_000000, startTime=1463524591920, finishTime=1463524591969, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1463524591710_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524591958, Process_Duration=6, Process_End_Time=1463524591964, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:31,970 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524591710_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:31,970 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602] : 1
2016-05-17 22:36:31,970 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602]
2016-05-17 22:36:31,971 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:31,971 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591710_0001_1][Event:VERTEX_FINISHED]: vertexName=A8B668D33A87492F91F41094C361D602, vertexId=vertex_1463524591710_0001_1_00, initRequestedTime=1463524591885, initedTime=1463524591907, startRequestedTime=1463524591908, startedTime=1463524591908, finishTime=1463524591970, timeTaken=62, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524591958, Process_Duration=6, Process_End_Time=1463524591964, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524591920, firstTasksToStart=[ task_1463524591710_0001_1_00_000000 ], lastTaskFinishTime=1463524591966, lastTasksToFinish=[ task_1463524591710_0001_1_00_000000 ], minTaskDuration=46, maxTaskDuration=46, avgTaskDuration=46.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524591710_0001_1_00_000000 ], longestDurationTasks=[ task_1463524591710_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:31,971 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:31,972 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524591710_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:31,972 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:31,972 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591710_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524591710_0001_00_000001, stoppedTime=1463524591972, exitStatus=0
2016-05-17 22:36:31,972 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524591710_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:31,972 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:31,972 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524591710_0001_1
2016-05-17 22:36:31,973 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591710_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524591710_0001_1
2016-05-17 22:36:31,973 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 7B95D1C6E8C34C6082C1BAC0D65FDE65 for vertex: vertex_1463524591710_0001_1_00 [A8B668D33A87492F91F41094C361D602]
2016-05-17 22:36:31,980 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524591710_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:31,981 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524591710_0001_00_000001: 13 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:31,981 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524591710_0001_00_000001, Exiting
2016-05-17 22:36:31,981 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524591710_0001_00_000001
2016-05-17 22:36:31,981 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524591710_0001_00_000001
2016-05-17 22:36:31,983 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524591710_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:31,983 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524591710_0001_00_000001 completed successfully
2016-05-17 22:36:31,990 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524591710_0001_1, queueSize=0
2016-05-17 22:36:31,990 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:31,990 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524591710_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524591710_0001_1, startTime=1463524591884, finishTime=1463524591989, timeTaken=105, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524591958, Process_Duration=6, Process_End_Time=1463524591964, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:31,990 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524591710_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:31,991 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524591710_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:31,991 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524591710_0001_00_000001
2016-05-17 22:36:31,991 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524591710_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:31 Completed Dag: dag_1463524591710_0001_1
2016-05-17 22:36:31,991 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524591710_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:31,991 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524591710_0001_1_post
2016-05-17 22:36:31,991 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:31,992 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:31,998 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/-1_UNKNOWN_SWAP, applicationId=application_1463524591710_0001
2016-05-17 22:36:32,001 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:32,001 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:32,001 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:32,001 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:32,001 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:32,002 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:32,003 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:32,003 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:32,003 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:32,004 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:32,004 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:32,005 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:32,005 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:32,009 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/76B73064343B42D5AE3288D21D90EC7D/D8DC6232DCD2446481F80D6DA83465CF/.tez/application_1463524591710_0001
2016-05-17 22:36:32,009 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_UNKNOWN_SWAP/_temporary
2016-05-17 22:36:32,010 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.321
2016-05-17 22:36:32,084 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_UNKNOWN_SWAP
2016-05-17 22:36:32,106 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:32,106 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:32,144 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 22:36:32,152 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-05-17 22:36:32,153 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:32,153 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:32,153 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:32,168 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:32,168 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_-1
2016-05-17 22:36:32,178 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:32,178 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:32,179 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_-1"]
2016-05-17 22:36:32,179 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:32,179 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:32,179 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:32,181 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...mblyplatform/-1_VALUES_-1
2016-05-17 22:36:32,181 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:32,193 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:32,197 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:32,197 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:32,198 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:32,200 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:32,200 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:32,207 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DD01FE2212294F91B44D13D2668DE684/DE58812503D64626A885AAECDC553C38/.tez/application_1463524592197_0001 doesn't exist and is created
2016-05-17 22:36:32,224 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:32,224 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:32,225 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DD01FE2212294F91B44D13D2668DE684/DE58812503D64626A885AAECDC553C38/.tez/application_1463524592197_0001
2016-05-17 22:36:32,229 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524592197_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:32,241 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:32,242 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:32,243 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:32,243 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:32,243 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DD01FE2212294F91B44D13D2668DE684/DE58812503D64626A885AAECDC553C38/.tez/application_1463524592197_0001/localmode-log-dir
2016-05-17 22:36:32,244 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DD01FE2212294F91B44D13D2668DE684/DE58812503D64626A885AAECDC553C38/.tez/application_1463524592197_0001/localmode-log-dir/history.txt.appattempt_1463524592197_0001_000000, maxErrors=10
2016-05-17 22:36:32,244 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:32,244 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524592197_0001_000000, appSubmitTime=1463524592229, launchTime=1463524592229
2016-05-17 22:36:32,249 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:32,249 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:32,251 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:32,251 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:32,252 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:44962
2016-05-17 22:36:32,256 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:32,258 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524592197_0001_000000, startTime=1463524592258
2016-05-17 22:36:32,258 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:32,258 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:32,325 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:32,325 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:32,325 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...mblyplatform/-1_VALUES_-1, applicationId=application_1463524592197_0001, dagName=[DD01FE2212294F91B44D13D2668DE684/DE58812503D64626A885AAECDC553C38] each/(1/1) ...mblyplatform/-1_VALUES_-1
2016-05-17 22:36:32,342 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [DD01FE2212294F91B44D13D2668DE684/DE58812503D64626A885AAECDC553C38] each/(1/1) ...mblyplatform/-1_VALUES_-1
2016-05-17 22:36:32,344 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524592197_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DD01FE2212294F91B44D13D2668DE684/DE58812503D64626A885AAECDC553C38/.tez/application_1463524592197_0001/localmode-log-dir/dag_1463524592197_0001_1.dot
2016-05-17 22:36:32,344 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DD01FE2212294F91B44D13D2668DE684/DE58812503D64626A885AAECDC553C38/.tez/application_1463524592197_0001/localmode-log-dir/dag_1463524592197_0001_1-tez-dag.pb.txt
2016-05-17 22:36:32,351 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524592197_0001_1
2016-05-17 22:36:32,351 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:32,351 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [DD01FE2212294F91B44D13D2668DE684/DE58812503D64626A885AAECDC553C38] each/(1/1) ...mblyplatform/-1_VALUES_-1
2016-05-17 22:36:32 Running Dag: dag_1463524592197_0001_1
2016-05-17 22:36:32,364 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592197_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524592197_0001_1, submitTime=1463524592342
2016-05-17 22:36:32,364 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:32,365 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex EECF87D25A154BE8A804039F57A003F3
2016-05-17 22:36:32,365 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:32,365 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592197_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524592197_0001_1, initTime=1463524592364
2016-05-17 22:36:32,365 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524592197_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:32,365 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:32,368 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592197_0001_1][Event:DAG_STARTED]: dagID=dag_1463524592197_0001_1, startTime=1463524592367
2016-05-17 22:36:32,368 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...mblyplatform/-1_VALUES_-1, applicationId=application_1463524592197_0001, dagName=[DD01FE2212294F91B44D13D2668DE684/DE58812503D64626A885AAECDC553C38] each/(1/1) ...mblyplatform/-1_VALUES_-1
2016-05-17 22:36:32,368 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:32,368 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:32,370 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524592197_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:32,370 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: EECF87D25A154BE8A804039F57A003F3 : {428703B7A2E74DCB9AC9A2EE238F876A={InputName=428703B7A2E74DCB9AC9A2EE238F876A}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:32,371 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 428703B7A2E74DCB9AC9A2EE238F876A, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:32,371 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3]
2016-05-17 22:36:32,371 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3]
2016-05-17 22:36:32,371 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3]
2016-05-17 22:36:32,371 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3]. Starting root input initializers: 1
2016-05-17 22:36:32,375 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:32,376 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 428703B7A2E74DCB9AC9A2EE238F876A on vertex vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3]
2016-05-17 22:36:32,377 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:32,379 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 428703B7A2E74DCB9AC9A2EE238F876A asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:32,380 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:32,381 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:32,383 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:32,384 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 428703B7A2E74DCB9AC9A2EE238F876A on vertex vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3]
2016-05-17 22:36:32,384 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:32,384 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {428703B7A2E74DCB9AC9A2EE238F876A=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:32,384 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3] parallelism set to 1
2016-05-17 22:36:32,385 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:32,385 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592197_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524592197_0001_1_00, eventCount=1
2016-05-17 22:36:32,385 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3]
2016-05-17 22:36:32,385 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=EE3CC55231EE4B9CB73E0C702AE3CF46, vertexId=vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:32,385 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=EE3CC55231EE4B9CB73E0C702AE3CF46, vertexId=vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3]
2016-05-17 22:36:32,389 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524592197_0001, with dag id: dag_1463524592197_0001_1
2016-05-17 22:36:32,391 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=EE3CC55231EE4B9CB73E0C702AE3CF46, vertexName=EECF87D25A154BE8A804039F57A003F3, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:32,392 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=EE3CC55231EE4B9CB73E0C702AE3CF46, vertexId=vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3]
2016-05-17 22:36:32,393 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592197_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EECF87D25A154BE8A804039F57A003F3, vertexId=vertex_1463524592197_0001_1_00, initRequestedTime=1463524592370, initedTime=1463524592393, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:32,393 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:32,393 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3] with distanceFromRoot: 0
2016-05-17 22:36:32,393 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:32,394 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in EECF87D25A154BE8A804039F57A003F3
2016-05-17 22:36:32,394 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3]
2016-05-17 22:36:32,395 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592197_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524592197_0001_1_00, startRequestedTime=1463524592394, startedTime=1463524592394
2016-05-17 22:36:32,395 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:32,395 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:32,395 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:32,396 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592197_0001_1][Event:TASK_STARTED]: vertexName=EECF87D25A154BE8A804039F57A003F3, taskId=task_1463524592197_0001_1_00_000000, scheduledTime=1463524592395, launchTime=1463524592395
2016-05-17 22:36:32,396 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524592197_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:32,397 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3]
2016-05-17 22:36:32,397 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [DD01FE2212294F91B44D13D2668DE684/DE58812503D64626A885AAECDC553C38] each/(1/1) ...mblyplatform/-1_VALUES_-1, VertexName: EECF87D25A154BE8A804039F57A003F3, VertexParallelism: 1, TaskAttemptID:attempt_1463524592197_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=428703B7A2E74DCB9AC9A2EE238F876A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EE3CC55231EE4B9CB73E0C702AE3CF46, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:32,398 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524592197_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:32,400 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:32,401 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:32,402 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:32,402 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:32,403 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524592197_0001_00_000001
2016-05-17 22:36:32,403 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524592197_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:32,405 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524592197_0001_00_000001
2016-05-17 22:36:32,405 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:32,406 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524592197_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:32,406 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:32,406 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592197_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524592197_0001_00_000001, launchTime=1463524592406
2016-05-17 22:36:32,407 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524592197_0001_00_000001
2016-05-17 22:36:32,408 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524592197_0001_1_00_000000_0] to container: [container_1463524592197_0001_00_000001]
2016-05-17 22:36:32,408 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524592197_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:32,409 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524592197_0001_1_00_000000_0] started. Is using containerId: [container_1463524592197_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:32,409 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592197_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EECF87D25A154BE8A804039F57A003F3, taskAttemptId=attempt_1463524592197_0001_1_00_000000_0, startTime=1463524592408, containerId=container_1463524592197_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524592197_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:32,409 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524592197_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:32,409 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524592197_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:32,410 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:32,410 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524592197_0001_00_000001 given task: attempt_1463524592197_0001_1_00_000000_0
2016-05-17 22:36:32,410 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524592197_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524592197_0001_1_00_000000_0
2016-05-17 22:36:32 Starting to run new task attempt: attempt_1463524592197_0001_1_00_000000_0
2016-05-17 22:36:32,411 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524592197_0001_1_00_000000_0
2016-05-17 22:36:32,411 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:32,411 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:32,412 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:32,412 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:32,412 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:32,412 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:32,412 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [DD01FE2212294F91B44D13D2668DE684/DE58812503D64626A885AAECDC553C38] each/(1/1) ...mblyplatform/-1_VALUES_-1, VertexName: EECF87D25A154BE8A804039F57A003F3, VertexParallelism: 1, TaskAttemptID:attempt_1463524592197_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=428703B7A2E74DCB9AC9A2EE238F876A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EE3CC55231EE4B9CB73E0C702AE3CF46, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:32,412 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:32,413 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524592197_0001_1_00_000000_0
2016-05-17 22:36:32,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:32,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:32,415 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=428703B7A2E74DCB9AC9A2EE238F876A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:32,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:32,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 428703B7A2E74DCB9AC9A2EE238F876A
2016-05-17 22:36:32,417 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 428703B7A2E74DCB9AC9A2EE238F876A
2016-05-17 22:36:32,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:32,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EE3CC55231EE4B9CB73E0C702AE3CF46, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:32,420 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:32,420 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EE3CC55231EE4B9CB73E0C702AE3CF46
2016-05-17 22:36:32,420 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:32,420 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: EE3CC55231EE4B9CB73E0C702AE3CF46
2016-05-17 22:36:32,422 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:32,426 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524592197_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:32,427 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:32,427 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 428703B7A2E74DCB9AC9A2EE238F876A
2016-05-17 22:36:32,427 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 428703B7A2E74DCB9AC9A2EE238F876A
2016-05-17 22:36:32,438 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EECF87D25A154BE8A804039F57A003F3, ordinal: 0
2016-05-17 22:36:32,438 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EECF87D25A154BE8A804039F57A003F3, mem on start (mb), free: 193, total: 388, max: 672
2016-05-17 22:36:32,438 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:32,439 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:32,439 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:32,442 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:32,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EE3CC55231EE4B9CB73E0C702AE3CF46
2016-05-17 22:36:32,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:32,443 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:32,443 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:32,444 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:32,444 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:32,444 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:32,444 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:32,445 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:32,445 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 428703B7A2E74DCB9AC9A2EE238F876A, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:32,445 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EE3CC55231EE4B9CB73E0C702AE3CF46, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:32,446 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:32,446 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 428703B7A2E74DCB9AC9A2EE238F876A being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:32,446 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 428703B7A2E74DCB9AC9A2EE238F876A
2016-05-17 22:36:32,446 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:32,446 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 428703B7A2E74DCB9AC9A2EE238F876A
2016-05-17 22:36:32,446 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:32,447 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:32,447 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:32,447 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524592197_0001_1_00_000000_0
2016-05-17 22:36:32,449 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:32,450 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@379b8cc2
2016-05-17 22:36:32,451 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:32,451 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@379b8cc2
2016-05-17 22:36:32,451 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 428703B7A2E74DCB9AC9A2EE238F876A
2016-05-17 22:36:32,452 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_-1"], id: EE3CC55231EE4B9CB73E0C702AE3CF46
2016-05-17 22:36:32,452 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:32,452 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_-1"]
2016-05-17 22:36:32,453 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:32,453 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:32,452 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:32,454 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EECF87D25A154BE8A804039F57A003F3, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:32,454 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:32,455 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:32,457 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:32,457 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:32,457 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524592197_0001_1_00_000000_0
2016-05-17 22:36:32,457 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524592197_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:32,460 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524592197_0001_1_00_000000_0
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:32,466 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592197_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EECF87D25A154BE8A804039F57A003F3, taskAttemptId=attempt_1463524592197_0001_1_00_000000_0, startTime=1463524592408, finishTime=1463524592466, timeTaken=58, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524592454, Process_Duration=6, Process_End_Time=1463524592460, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:36:32,467 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524592197_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:36:32,467 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524592197_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:36:32,467 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592197_0001_1][Event:TASK_FINISHED]: vertexName=EECF87D25A154BE8A804039F57A003F3, taskId=task_1463524592197_0001_1_00_000000, startTime=1463524592408, finishTime=1463524592467, timeTaken=59, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
], counters=Counters: 0
2016-05-17 22:36:32,467 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524592197_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:36:32,468 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:32,468 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3] : 1
2016-05-17 22:36:32,469 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3] because task failed: task_1463524592197_0001_1_00_000000
2016-05-17 22:36:32,469 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:36:32,469 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:32,469 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:36:32,470 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3]
2016-05-17 22:36:32,470 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=EE3CC55231EE4B9CB73E0C702AE3CF46, vertexId=vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3]
2016-05-17 22:36:32,470 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:32,473 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592197_0001_1][Event:VERTEX_FINISHED]: vertexName=EECF87D25A154BE8A804039F57A003F3, vertexId=vertex_1463524592197_0001_1_00, initRequestedTime=1463524592370, initedTime=1463524592393, startRequestedTime=1463524592394, startedTime=1463524592394, finishTime=1463524592471, timeTaken=77, status=FAILED, diagnostics=Task failed, taskId=task_1463524592197_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524592408, firstTasksToStart=[ task_1463524592197_0001_1_00_000000 ], lastTaskFinishTime=1463524592466, lastTasksToFinish=[ task_1463524592197_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:36:32,474 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:36:32,476 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592197_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524592197_0001_00_000001, stoppedTime=1463524592476, exitStatus=0
2016-05-17 22:36:32,476 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524592197_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:32,476 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:32,476 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524592197_0001_1_00_000000_0 failed: true container: container_1463524592197_0001_00_000001 numFailedTAs: 0
2016-05-17 22:36:32,481 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:36:32,481 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:36:32,481 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:32,482 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524592197_0001_1
2016-05-17 22:36:32,482 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: EE3CC55231EE4B9CB73E0C702AE3CF46 for vertex: vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3]
2016-05-17 22:36:32,483 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524592197_0001_1, queueSize=0
2016-05-17 22:36:32,483 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 19 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524592454, Process_Duration=6, Process_End_Time=1463524592460, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:36:32,483 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EECF87D25A154BE8A804039F57A003F3, mem on close (mb), free: 191, total: 388, max: 672
2016-05-17 22:36:32,483 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=EECF87D25A154BE8A804039F57A003F3, index=0
2016-05-17 22:36:32,483 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=EECF87D25A154BE8A804039F57A003F3, sourceVertex=428703B7A2E74DCB9AC9A2EE238F876A
2016-05-17 22:36:32,483 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=EECF87D25A154BE8A804039F57A003F3, sourceVertex=EE3CC55231EE4B9CB73E0C702AE3CF46
2016-05-17 22:36:32,484 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:32,484 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592197_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524592197_0001_1, startTime=1463524592367, finishTime=1463524592482, timeTaken=115, status=FAILED, diagnostics=Vertex failed, vertexName=EECF87D25A154BE8A804039F57A003F3, vertexId=vertex_1463524592197_0001_1_00, diagnostics=[Task failed, taskId=task_1463524592197_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524592197_0001_1_00 [EECF87D25A154BE8A804039F57A003F3] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:36:32,485 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524592197_0001_00_000001
2016-05-17 22:36:32,485 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524592197_0001_00_000001
2016-05-17 22:36:32,487 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524592197_0001_1 finished with state: FAILED
2016-05-17 22:36:32,487 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524592197_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:32,488 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524592197_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:36:32 Completed Dag: dag_1463524592197_0001_1
2016-05-17 22:36:32,489 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524592197_0001_1, dagState=FAILED
2016-05-17 22:36:32,489 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524592197_0001_1_post
2016-05-17 22:36:32,489 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:32,489 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:32,489 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524592197_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:36:32,491 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524592197_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:36:32,491 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524592197_0001_00_000001 completed but with errors
2016-05-17 22:36:32,492 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524592197_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:36:32,492 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524592197_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:36:32,492 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524592197_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:36:32,496 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...mblyplatform/-1_VALUES_-1, applicationId=application_1463524592197_0001
2016-05-17 22:36:32,504 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:32,504 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:32,504 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:32,504 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:32,504 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:32,505 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:32,505 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:32,505 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:32,506 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:32,506 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:32,506 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:32,506 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:32,507 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:32,510 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DD01FE2212294F91B44D13D2668DE684/DE58812503D64626A885AAECDC553C38/.tez/application_1463524592197_0001
2016-05-17 22:36:32,510 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping all jobs
2016-05-17 22:36:32,511 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping: (1/1) ...mblyplatform/-1_VALUES_-1
2016-05-17 22:36:32,511 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopped all jobs
2016-05-17 22:36:32,511 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_-1/_temporary
2016-05-17 22:36:32,512 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.333
2016-05-17 22:36:32,512 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_VALUES_-1
2016-05-17 22:36:32,536 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:32,536 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:32,566 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-05-17 22:36:32,569 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.008
2016-05-17 22:36:32,570 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:32,570 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:32,570 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_VALUES_'field'
2016-05-17 22:36:32,595 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:32,596 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:32,634 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 22:36:32,640 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-17 22:36:32,641 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:32,641 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:32,641 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:32,656 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:32,657 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_RESULTS
2016-05-17 22:36:32,666 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:32,667 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:32,667 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_RESULTS"]
2016-05-17 22:36:32,667 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:32,667 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:32,668 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:32,668 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...latform/-1_VALUES_RESULTS
2016-05-17 22:36:32,669 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:32,680 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:32,685 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:32,685 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:32,685 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:32,695 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6F5DC91FD91E4E63ACCE820B970DABF2/B0EF617C85E74C3F8AC34264814167B9/.tez/application_1463524592685_0001 doesn't exist and is created
2016-05-17 22:36:32,713 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:32,713 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:32,714 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6F5DC91FD91E4E63ACCE820B970DABF2/B0EF617C85E74C3F8AC34264814167B9/.tez/application_1463524592685_0001
2016-05-17 22:36:32,717 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524592685_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:32,727 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:32,727 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:32,729 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:32,729 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:32,729 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6F5DC91FD91E4E63ACCE820B970DABF2/B0EF617C85E74C3F8AC34264814167B9/.tez/application_1463524592685_0001/localmode-log-dir
2016-05-17 22:36:32,730 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6F5DC91FD91E4E63ACCE820B970DABF2/B0EF617C85E74C3F8AC34264814167B9/.tez/application_1463524592685_0001/localmode-log-dir/history.txt.appattempt_1463524592685_0001_000000, maxErrors=10
2016-05-17 22:36:32,730 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:32,730 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524592685_0001_000000, appSubmitTime=1463524592717, launchTime=1463524592717
2016-05-17 22:36:32,748 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:32,750 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:32,752 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:32,753 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:32,755 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:46181
2016-05-17 22:36:32,757 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:32,761 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524592685_0001_000000, startTime=1463524592761
2016-05-17 22:36:32,761 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:32,761 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:32,814 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:32,814 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:32,814 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/-1_VALUES_RESULTS, applicationId=application_1463524592685_0001, dagName=[6F5DC91FD91E4E63ACCE820B970DABF2/B0EF617C85E74C3F8AC34264814167B9] each/(1/1) ...latform/-1_VALUES_RESULTS
2016-05-17 22:36:32,831 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [6F5DC91FD91E4E63ACCE820B970DABF2/B0EF617C85E74C3F8AC34264814167B9] each/(1/1) ...latform/-1_VALUES_RESULTS
2016-05-17 22:36:32,834 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524592685_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6F5DC91FD91E4E63ACCE820B970DABF2/B0EF617C85E74C3F8AC34264814167B9/.tez/application_1463524592685_0001/localmode-log-dir/dag_1463524592685_0001_1.dot
2016-05-17 22:36:32,834 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6F5DC91FD91E4E63ACCE820B970DABF2/B0EF617C85E74C3F8AC34264814167B9/.tez/application_1463524592685_0001/localmode-log-dir/dag_1463524592685_0001_1-tez-dag.pb.txt
2016-05-17 22:36:32,840 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524592685_0001_1
2016-05-17 22:36:32,841 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:32,841 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [6F5DC91FD91E4E63ACCE820B970DABF2/B0EF617C85E74C3F8AC34264814167B9] each/(1/1) ...latform/-1_VALUES_RESULTS
2016-05-17 22:36:32 Running Dag: dag_1463524592685_0001_1
2016-05-17 22:36:32,853 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592685_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524592685_0001_1, submitTime=1463524592832
2016-05-17 22:36:32,853 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:32,854 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FD5431836AD14EE59FB49D0A75F05765
2016-05-17 22:36:32,854 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:32,855 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:32,855 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592685_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524592685_0001_1, initTime=1463524592854
2016-05-17 22:36:32,856 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524592685_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:32,856 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592685_0001_1][Event:DAG_STARTED]: dagID=dag_1463524592685_0001_1, startTime=1463524592856
2016-05-17 22:36:32,856 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:32,856 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:32,856 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524592685_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:32,857 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FD5431836AD14EE59FB49D0A75F05765 : {7F8D0216A36F4A5B89628A7E4C3BA916={InputName=7F8D0216A36F4A5B89628A7E4C3BA916}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:32,857 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 7F8D0216A36F4A5B89628A7E4C3BA916, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:32,857 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765]
2016-05-17 22:36:32,857 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/-1_VALUES_RESULTS, applicationId=application_1463524592685_0001, dagName=[6F5DC91FD91E4E63ACCE820B970DABF2/B0EF617C85E74C3F8AC34264814167B9] each/(1/1) ...latform/-1_VALUES_RESULTS
2016-05-17 22:36:32,857 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765]
2016-05-17 22:36:32,858 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765]
2016-05-17 22:36:32,858 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765]. Starting root input initializers: 1
2016-05-17 22:36:32,858 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:32,858 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 7F8D0216A36F4A5B89628A7E4C3BA916 on vertex vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765]
2016-05-17 22:36:32,861 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:32,862 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 7F8D0216A36F4A5B89628A7E4C3BA916 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:32,863 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:32,863 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:32,865 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:32,865 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 7F8D0216A36F4A5B89628A7E4C3BA916 on vertex vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765]
2016-05-17 22:36:32,866 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:32,866 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {7F8D0216A36F4A5B89628A7E4C3BA916=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:32,866 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765] parallelism set to 1
2016-05-17 22:36:32,866 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:32,867 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592685_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524592685_0001_1_00, eventCount=1
2016-05-17 22:36:32,867 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765]
2016-05-17 22:36:32,867 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=6AD8F62D477A47B2892711FE4B2DEB33, vertexId=vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:32,867 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=6AD8F62D477A47B2892711FE4B2DEB33, vertexId=vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765]
2016-05-17 22:36:32,868 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=6AD8F62D477A47B2892711FE4B2DEB33, vertexName=FD5431836AD14EE59FB49D0A75F05765, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:32,869 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=6AD8F62D477A47B2892711FE4B2DEB33, vertexId=vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765]
2016-05-17 22:36:32,870 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:32,870 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592685_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FD5431836AD14EE59FB49D0A75F05765, vertexId=vertex_1463524592685_0001_1_00, initRequestedTime=1463524592857, initedTime=1463524592870, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:32,871 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765] with distanceFromRoot: 0
2016-05-17 22:36:32,871 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:32,871 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FD5431836AD14EE59FB49D0A75F05765
2016-05-17 22:36:32,871 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765]
2016-05-17 22:36:32,871 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:32,872 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592685_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524592685_0001_1_00, startRequestedTime=1463524592871, startedTime=1463524592871
2016-05-17 22:36:32,872 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:32,872 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:32,872 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592685_0001_1][Event:TASK_STARTED]: vertexName=FD5431836AD14EE59FB49D0A75F05765, taskId=task_1463524592685_0001_1_00_000000, scheduledTime=1463524592872, launchTime=1463524592872
2016-05-17 22:36:32,873 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524592685_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:32,873 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765]
2016-05-17 22:36:32,873 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [6F5DC91FD91E4E63ACCE820B970DABF2/B0EF617C85E74C3F8AC34264814167B9] each/(1/1) ...latform/-1_VALUES_RESULTS, VertexName: FD5431836AD14EE59FB49D0A75F05765, VertexParallelism: 1, TaskAttemptID:attempt_1463524592685_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7F8D0216A36F4A5B89628A7E4C3BA916, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=6AD8F62D477A47B2892711FE4B2DEB33, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:32,874 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524592685_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:32,874 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:32,875 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:32,875 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:32,875 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:32,876 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524592685_0001_00_000001
2016-05-17 22:36:32,876 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524592685_0001, with dag id: dag_1463524592685_0001_1
2016-05-17 22:36:32,876 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524592685_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:32,877 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524592685_0001_00_000001
2016-05-17 22:36:32,878 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:32,878 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524592685_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:32,879 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:32,879 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592685_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524592685_0001_00_000001, launchTime=1463524592878
2016-05-17 22:36:32,880 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524592685_0001_00_000001
2016-05-17 22:36:32,880 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524592685_0001_1_00_000000_0] to container: [container_1463524592685_0001_00_000001]
2016-05-17 22:36:32,880 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524592685_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:32,882 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524592685_0001_00_000001 given task: attempt_1463524592685_0001_1_00_000000_0
2016-05-17 22:36:32,883 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524592685_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524592685_0001_1_00_000000_0
2016-05-17 22:36:32,883 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524592685_0001_1_00_000000_0] started. Is using containerId: [container_1463524592685_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:32,883 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592685_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FD5431836AD14EE59FB49D0A75F05765, taskAttemptId=attempt_1463524592685_0001_1_00_000000_0, startTime=1463524592883, containerId=container_1463524592685_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524592685_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:32 Starting to run new task attempt: attempt_1463524592685_0001_1_00_000000_0
2016-05-17 22:36:32,883 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524592685_0001_1_00_000000_0
2016-05-17 22:36:32,883 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524592685_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:32,884 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:32,884 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524592685_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:32,884 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:32,884 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:32,884 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:32,884 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:32,884 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:32,884 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:32,884 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6F5DC91FD91E4E63ACCE820B970DABF2/B0EF617C85E74C3F8AC34264814167B9] each/(1/1) ...latform/-1_VALUES_RESULTS, VertexName: FD5431836AD14EE59FB49D0A75F05765, VertexParallelism: 1, TaskAttemptID:attempt_1463524592685_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7F8D0216A36F4A5B89628A7E4C3BA916, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=6AD8F62D477A47B2892711FE4B2DEB33, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:32,885 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:32,885 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524592685_0001_1_00_000000_0
2016-05-17 22:36:32,885 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:32,887 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:32,888 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524592685_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:32,888 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=7F8D0216A36F4A5B89628A7E4C3BA916, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:32,889 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:32,889 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 7F8D0216A36F4A5B89628A7E4C3BA916
2016-05-17 22:36:32,889 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 7F8D0216A36F4A5B89628A7E4C3BA916
2016-05-17 22:36:32,890 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:32,890 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=6AD8F62D477A47B2892711FE4B2DEB33, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:32,890 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:32,891 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:32,891 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:32,891 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 6AD8F62D477A47B2892711FE4B2DEB33
2016-05-17 22:36:32,891 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:32,891 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 6AD8F62D477A47B2892711FE4B2DEB33
2016-05-17 22:36:32,893 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:32,898 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:32,898 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 7F8D0216A36F4A5B89628A7E4C3BA916
2016-05-17 22:36:32,898 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 7F8D0216A36F4A5B89628A7E4C3BA916
2016-05-17 22:36:32,903 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FD5431836AD14EE59FB49D0A75F05765, ordinal: 0
2016-05-17 22:36:32,904 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FD5431836AD14EE59FB49D0A75F05765, mem on start (mb), free: 327, total: 388, max: 672
2016-05-17 22:36:32,904 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:32,904 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:32,905 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:32,911 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:32,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 6AD8F62D477A47B2892711FE4B2DEB33
2016-05-17 22:36:32,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:32,912 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:32,912 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:32,913 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:32,913 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:32,914 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:32,914 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:32,915 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:32,915 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 7F8D0216A36F4A5B89628A7E4C3BA916, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:32,915 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 6AD8F62D477A47B2892711FE4B2DEB33, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:32,916 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:32,916 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 7F8D0216A36F4A5B89628A7E4C3BA916 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:32,916 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 7F8D0216A36F4A5B89628A7E4C3BA916
2016-05-17 22:36:32,917 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:32,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 7F8D0216A36F4A5B89628A7E4C3BA916
2016-05-17 22:36:32,916 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:32,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:32,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:32,917 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524592685_0001_1_00_000000_0
2016-05-17 22:36:32,918 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:32,918 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@523130fa
2016-05-17 22:36:32,920 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:32,921 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@523130fa
2016-05-17 22:36:32,921 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:32,921 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:32,921 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 7F8D0216A36F4A5B89628A7E4C3BA916
2016-05-17 22:36:32,922 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_RESULTS"], id: 6AD8F62D477A47B2892711FE4B2DEB33
2016-05-17 22:36:32,923 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_RESULTS"]
2016-05-17 22:36:32,924 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:32,924 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:32,925 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FD5431836AD14EE59FB49D0A75F05765, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:32,925 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:32,934 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:32,942 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:32,942 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:32,944 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524592685_0001_1_00_000000_0
2016-05-17 22:36:32,944 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524592685_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:32,945 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524592685_0001_1_00_000000_0
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:32,954 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407371776, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524592925, Process_Duration=20, Process_End_Time=1463524592945, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:36:32,955 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592685_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FD5431836AD14EE59FB49D0A75F05765, taskAttemptId=attempt_1463524592685_0001_1_00_000000_0, startTime=1463524592883, finishTime=1463524592955, timeTaken=72, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407371776, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524592925, Process_Duration=20, Process_End_Time=1463524592945, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:36:32,955 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524592685_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:36:32,956 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524592685_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:36:32,956 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592685_0001_1][Event:TASK_FINISHED]: vertexName=FD5431836AD14EE59FB49D0A75F05765, taskId=task_1463524592685_0001_1_00_000000, startTime=1463524592883, finishTime=1463524592956, timeTaken=73, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
], counters=Counters: 0
2016-05-17 22:36:32,955 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:32,956 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524592685_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:36:32,957 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765] : 1
2016-05-17 22:36:32,957 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765] because task failed: task_1463524592685_0001_1_00_000000
2016-05-17 22:36:32,957 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:36:32,957 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:36:32,957 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:32,956 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FD5431836AD14EE59FB49D0A75F05765, mem on close (mb), free: 324, total: 388, max: 672
2016-05-17 22:36:32,957 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765]
2016-05-17 22:36:32,958 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=6AD8F62D477A47B2892711FE4B2DEB33, vertexId=vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765]
2016-05-17 22:36:32,957 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:32,960 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:32,960 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592685_0001_1][Event:VERTEX_FINISHED]: vertexName=FD5431836AD14EE59FB49D0A75F05765, vertexId=vertex_1463524592685_0001_1_00, initRequestedTime=1463524592857, initedTime=1463524592870, startRequestedTime=1463524592871, startedTime=1463524592871, finishTime=1463524592959, timeTaken=88, status=FAILED, diagnostics=Task failed, taskId=task_1463524592685_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524592883, firstTasksToStart=[ task_1463524592685_0001_1_00_000000 ], lastTaskFinishTime=1463524592954, lastTasksToFinish=[ task_1463524592685_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:36:32,961 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:36:32,961 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:36:32,962 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:36:32,962 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524592685_0001_1
2016-05-17 22:36:32,962 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 6AD8F62D477A47B2892711FE4B2DEB33 for vertex: vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765]
2016-05-17 22:36:32,963 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524592685_0001_1, queueSize=0
2016-05-17 22:36:32,963 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:32,964 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592685_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524592685_0001_1, startTime=1463524592856, finishTime=1463524592962, timeTaken=106, status=FAILED, diagnostics=Vertex failed, vertexName=FD5431836AD14EE59FB49D0A75F05765, vertexId=vertex_1463524592685_0001_1_00, diagnostics=[Task failed, taskId=task_1463524592685_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524592685_0001_1_00 [FD5431836AD14EE59FB49D0A75F05765] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:36:32,965 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524592685_0001_1 finished with state: FAILED
2016-05-17 22:36:32,965 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524592685_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:32,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524592685_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524592685_0001_00_000001, stoppedTime=1463524592966, exitStatus=0
2016-05-17 22:36:32,968 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:32,968 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524592685_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:32,968 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524592685_0001_1_00_000000_0 failed: true container: container_1463524592685_0001_00_000001 numFailedTAs: 0
2016-05-17 22:36:32 Completed Dag: dag_1463524592685_0001_1
2016-05-17 22:36:32,970 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524592685_0001_1, dagState=FAILED
2016-05-17 22:36:32,970 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524592685_0001_1_post
2016-05-17 22:36:32,970 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:32,970 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:32,971 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=FD5431836AD14EE59FB49D0A75F05765, index=0
2016-05-17 22:36:32,971 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=FD5431836AD14EE59FB49D0A75F05765, sourceVertex=7F8D0216A36F4A5B89628A7E4C3BA916
2016-05-17 22:36:32,971 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=FD5431836AD14EE59FB49D0A75F05765, sourceVertex=6AD8F62D477A47B2892711FE4B2DEB33
2016-05-17 22:36:32,971 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524592685_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:36:32,971 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524592685_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:36:32,972 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524592685_0001_00_000001
2016-05-17 22:36:32,976 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524592685_0001_00_000001
2016-05-17 22:36:32,978 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524592685_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:36:32,982 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524592685_0001_00_000001 completed but with errors
2016-05-17 22:36:32,982 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524592685_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:36:32,982 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524592685_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:36:32,981 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/-1_VALUES_RESULTS, applicationId=application_1463524592685_0001
2016-05-17 22:36:32,982 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524592685_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:36:32,985 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:32,985 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:32,985 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:32,985 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:32,985 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:32,986 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:32,986 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:32,986 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:32,986 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:32,987 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:32,987 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:32,987 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:32,987 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:32,992 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6F5DC91FD91E4E63ACCE820B970DABF2/B0EF617C85E74C3F8AC34264814167B9/.tez/application_1463524592685_0001
2016-05-17 22:36:32,993 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping all jobs
2016-05-17 22:36:32,993 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping: (1/1) ...latform/-1_VALUES_RESULTS
2016-05-17 22:36:32,993 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopped all jobs
2016-05-17 22:36:32,994 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_RESULTS/_temporary
2016-05-17 22:36:32,994 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.328
2016-05-17 22:36:32,994 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_VALUES_RESULTS
2016-05-17 22:36:33,023 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:33,024 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:33,070 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 22:36:33,078 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-17 22:36:33,078 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:33,078 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:33,078 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:33,092 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:33,093 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_ALL
2016-05-17 22:36:33,104 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:33,104 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:33,104 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_ALL"]
2016-05-17 22:36:33,105 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:33,105 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:33,105 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:33,105 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...blyplatform/-1_VALUES_ALL
2016-05-17 22:36:33,106 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:33,119 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:33,124 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:33,124 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:33,124 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:33,134 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8C9887C9EC164FFFA1199073D6A50527/A7153FC73C874DDEB10F793D6D1FF3EF/.tez/application_1463524593124_0001 doesn't exist and is created
2016-05-17 22:36:33,152 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:33,152 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:33,153 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8C9887C9EC164FFFA1199073D6A50527/A7153FC73C874DDEB10F793D6D1FF3EF/.tez/application_1463524593124_0001
2016-05-17 22:36:33,157 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524593124_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:33,168 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:33,168 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:33,170 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:33,171 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:33,171 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8C9887C9EC164FFFA1199073D6A50527/A7153FC73C874DDEB10F793D6D1FF3EF/.tez/application_1463524593124_0001/localmode-log-dir
2016-05-17 22:36:33,171 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8C9887C9EC164FFFA1199073D6A50527/A7153FC73C874DDEB10F793D6D1FF3EF/.tez/application_1463524593124_0001/localmode-log-dir/history.txt.appattempt_1463524593124_0001_000000, maxErrors=10
2016-05-17 22:36:33,171 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:33,172 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524593124_0001_000000, appSubmitTime=1463524593156, launchTime=1463524593156
2016-05-17 22:36:33,175 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:33,178 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:33,179 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:40938
2016-05-17 22:36:33,181 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:33,181 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:33,182 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:33,184 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524593124_0001_000000, startTime=1463524593184
2016-05-17 22:36:33,184 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:33,184 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:33,253 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:33,253 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:33,253 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/-1_VALUES_ALL, applicationId=application_1463524593124_0001, dagName=[8C9887C9EC164FFFA1199073D6A50527/A7153FC73C874DDEB10F793D6D1FF3EF] each/(1/1) ...blyplatform/-1_VALUES_ALL
2016-05-17 22:36:33,272 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [8C9887C9EC164FFFA1199073D6A50527/A7153FC73C874DDEB10F793D6D1FF3EF] each/(1/1) ...blyplatform/-1_VALUES_ALL
2016-05-17 22:36:33,274 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524593124_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8C9887C9EC164FFFA1199073D6A50527/A7153FC73C874DDEB10F793D6D1FF3EF/.tez/application_1463524593124_0001/localmode-log-dir/dag_1463524593124_0001_1.dot
2016-05-17 22:36:33,274 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8C9887C9EC164FFFA1199073D6A50527/A7153FC73C874DDEB10F793D6D1FF3EF/.tez/application_1463524593124_0001/localmode-log-dir/dag_1463524593124_0001_1-tez-dag.pb.txt
2016-05-17 22:36:33,281 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524593124_0001_1
2016-05-17 22:36:33,282 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:33,282 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [8C9887C9EC164FFFA1199073D6A50527/A7153FC73C874DDEB10F793D6D1FF3EF] each/(1/1) ...blyplatform/-1_VALUES_ALL
2016-05-17 22:36:33 Running Dag: dag_1463524593124_0001_1
2016-05-17 22:36:33,294 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593124_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524593124_0001_1, submitTime=1463524593272
2016-05-17 22:36:33,294 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:33,295 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E7F6CAB4A5434CC6A62E050737157993
2016-05-17 22:36:33,295 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:33,296 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593124_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524593124_0001_1, initTime=1463524593295
2016-05-17 22:36:33,296 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:33,296 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524593124_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:33,297 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593124_0001_1][Event:DAG_STARTED]: dagID=dag_1463524593124_0001_1, startTime=1463524593297
2016-05-17 22:36:33,297 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:33,297 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/-1_VALUES_ALL, applicationId=application_1463524593124_0001, dagName=[8C9887C9EC164FFFA1199073D6A50527/A7153FC73C874DDEB10F793D6D1FF3EF] each/(1/1) ...blyplatform/-1_VALUES_ALL
2016-05-17 22:36:33,297 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:33,297 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524593124_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:33,298 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E7F6CAB4A5434CC6A62E050737157993 : {0DCCAEEABDBE43ECB61D39F1D021DF5A={InputName=0DCCAEEABDBE43ECB61D39F1D021DF5A}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:33,298 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 0DCCAEEABDBE43ECB61D39F1D021DF5A, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:33,299 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993]
2016-05-17 22:36:33,299 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993]
2016-05-17 22:36:33,299 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993]
2016-05-17 22:36:33,299 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993]. Starting root input initializers: 1
2016-05-17 22:36:33,300 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:33,300 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 0DCCAEEABDBE43ECB61D39F1D021DF5A on vertex vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993]
2016-05-17 22:36:33,302 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:33,302 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 0DCCAEEABDBE43ECB61D39F1D021DF5A asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:33,303 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:33,303 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:33,305 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:33,305 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 0DCCAEEABDBE43ECB61D39F1D021DF5A on vertex vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993]
2016-05-17 22:36:33,306 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:33,306 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {0DCCAEEABDBE43ECB61D39F1D021DF5A=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:33,306 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993] parallelism set to 1
2016-05-17 22:36:33,306 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:33,306 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593124_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524593124_0001_1_00, eventCount=1
2016-05-17 22:36:33,306 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993]
2016-05-17 22:36:33,306 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D7184FB845BB43FFAAB272F0AA9582FE, vertexId=vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:33,307 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D7184FB845BB43FFAAB272F0AA9582FE, vertexId=vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993]
2016-05-17 22:36:33,308 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D7184FB845BB43FFAAB272F0AA9582FE, vertexName=E7F6CAB4A5434CC6A62E050737157993, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:33,309 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D7184FB845BB43FFAAB272F0AA9582FE, vertexId=vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993]
2016-05-17 22:36:33,310 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593124_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E7F6CAB4A5434CC6A62E050737157993, vertexId=vertex_1463524593124_0001_1_00, initRequestedTime=1463524593298, initedTime=1463524593310, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:33,310 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993] with distanceFromRoot: 0
2016-05-17 22:36:33,310 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:33,310 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:33,310 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E7F6CAB4A5434CC6A62E050737157993
2016-05-17 22:36:33,311 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993]
2016-05-17 22:36:33,311 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593124_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524593124_0001_1_00, startRequestedTime=1463524593310, startedTime=1463524593310
2016-05-17 22:36:33,311 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:33,311 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:33,311 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:33,312 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593124_0001_1][Event:TASK_STARTED]: vertexName=E7F6CAB4A5434CC6A62E050737157993, taskId=task_1463524593124_0001_1_00_000000, scheduledTime=1463524593311, launchTime=1463524593311
2016-05-17 22:36:33,312 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524593124_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:33,312 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993]
2016-05-17 22:36:33,312 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8C9887C9EC164FFFA1199073D6A50527/A7153FC73C874DDEB10F793D6D1FF3EF] each/(1/1) ...blyplatform/-1_VALUES_ALL, VertexName: E7F6CAB4A5434CC6A62E050737157993, VertexParallelism: 1, TaskAttemptID:attempt_1463524593124_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0DCCAEEABDBE43ECB61D39F1D021DF5A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D7184FB845BB43FFAAB272F0AA9582FE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:33,313 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524593124_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:33,313 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:33,313 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:33,314 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:33,314 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:33,314 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524593124_0001, with dag id: dag_1463524593124_0001_1
2016-05-17 22:36:33,315 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524593124_0001_00_000001
2016-05-17 22:36:33,315 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524593124_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:33,318 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524593124_0001_00_000001
2016-05-17 22:36:33,318 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:33,319 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524593124_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:33,320 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:33,320 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593124_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524593124_0001_00_000001, launchTime=1463524593319
2016-05-17 22:36:33,321 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524593124_0001_00_000001
2016-05-17 22:36:33,322 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524593124_0001_1_00_000000_0] to container: [container_1463524593124_0001_00_000001]
2016-05-17 22:36:33,322 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524593124_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:33,322 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524593124_0001_1_00_000000_0] started. Is using containerId: [container_1463524593124_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:33,322 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:33,322 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593124_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E7F6CAB4A5434CC6A62E050737157993, taskAttemptId=attempt_1463524593124_0001_1_00_000000_0, startTime=1463524593322, containerId=container_1463524593124_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524593124_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:33,323 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524593124_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:33,323 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524593124_0001_00_000001 given task: attempt_1463524593124_0001_1_00_000000_0
2016-05-17 22:36:33,323 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524593124_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:33,323 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524593124_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524593124_0001_1_00_000000_0
2016-05-17 22:36:33 Starting to run new task attempt: attempt_1463524593124_0001_1_00_000000_0
2016-05-17 22:36:33,324 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524593124_0001_1_00_000000_0
2016-05-17 22:36:33,324 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:33,324 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:33,324 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:33,325 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:33,325 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:33,325 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:33,325 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8C9887C9EC164FFFA1199073D6A50527/A7153FC73C874DDEB10F793D6D1FF3EF] each/(1/1) ...blyplatform/-1_VALUES_ALL, VertexName: E7F6CAB4A5434CC6A62E050737157993, VertexParallelism: 1, TaskAttemptID:attempt_1463524593124_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0DCCAEEABDBE43ECB61D39F1D021DF5A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D7184FB845BB43FFAAB272F0AA9582FE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:33,325 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:33,326 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524593124_0001_1_00_000000_0
2016-05-17 22:36:33,326 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:33,326 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:33,327 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524593124_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:33,327 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=0DCCAEEABDBE43ECB61D39F1D021DF5A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:33,328 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:33,328 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 0DCCAEEABDBE43ECB61D39F1D021DF5A
2016-05-17 22:36:33,328 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 0DCCAEEABDBE43ECB61D39F1D021DF5A
2016-05-17 22:36:33,328 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:33,329 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D7184FB845BB43FFAAB272F0AA9582FE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:33,329 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:33,330 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D7184FB845BB43FFAAB272F0AA9582FE
2016-05-17 22:36:33,330 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:33,330 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D7184FB845BB43FFAAB272F0AA9582FE
2016-05-17 22:36:33,331 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:33,341 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:33,342 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 0DCCAEEABDBE43ECB61D39F1D021DF5A
2016-05-17 22:36:33,342 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 0DCCAEEABDBE43ECB61D39F1D021DF5A
2016-05-17 22:36:33,344 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E7F6CAB4A5434CC6A62E050737157993, ordinal: 0
2016-05-17 22:36:33,344 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E7F6CAB4A5434CC6A62E050737157993, mem on start (mb), free: 263, total: 388, max: 672
2016-05-17 22:36:33,344 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:33,344 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:33,345 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:33,347 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:33,347 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D7184FB845BB43FFAAB272F0AA9582FE
2016-05-17 22:36:33,347 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:33,348 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:33,348 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:33,348 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:33,348 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:33,348 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:33,349 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:33,349 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:33,349 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 0DCCAEEABDBE43ECB61D39F1D021DF5A, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:33,350 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D7184FB845BB43FFAAB272F0AA9582FE, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:33,350 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:33,351 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 0DCCAEEABDBE43ECB61D39F1D021DF5A being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:33,351 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 0DCCAEEABDBE43ECB61D39F1D021DF5A
2016-05-17 22:36:33,351 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:33,352 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 0DCCAEEABDBE43ECB61D39F1D021DF5A
2016-05-17 22:36:33,351 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:33,352 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:33,352 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:33,352 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524593124_0001_1_00_000000_0
2016-05-17 22:36:33,356 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:33,356 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 0DCCAEEABDBE43ECB61D39F1D021DF5A
2016-05-17 22:36:33,356 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_ALL"], id: D7184FB845BB43FFAAB272F0AA9582FE
2016-05-17 22:36:33,357 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_ALL"]
2016-05-17 22:36:33,357 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:33,357 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:33,357 INFO input.MRInput (MRInput.java:checkAndAwaitRecordReaderInitialization(597)) - Awaiting RecordReader initialization
2016-05-17 22:36:33,358 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:33,358 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7978def7
2016-05-17 22:36:33,360 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7978def7
2016-05-17 22:36:33,360 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:33,361 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:33,361 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E7F6CAB4A5434CC6A62E050737157993, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:33,362 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:33,365 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:33,365 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:33,366 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:33,366 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524593124_0001_1_00_000000_0
2016-05-17 22:36:33,366 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524593124_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:33,366 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524593124_0001_1_00_000000_0
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:33,371 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407371776, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524593361, Process_Duration=5, Process_End_Time=1463524593366, Read_Duration=1, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:36:33,371 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E7F6CAB4A5434CC6A62E050737157993, mem on close (mb), free: 261, total: 388, max: 672
2016-05-17 22:36:33,371 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593124_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E7F6CAB4A5434CC6A62E050737157993, taskAttemptId=attempt_1463524593124_0001_1_00_000000_0, startTime=1463524593322, finishTime=1463524593371, timeTaken=49, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407371776, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524593361, Process_Duration=5, Process_End_Time=1463524593366, Read_Duration=1, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:36:33,372 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524593124_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:36:33,372 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524593124_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:36:33,372 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593124_0001_1][Event:TASK_FINISHED]: vertexName=E7F6CAB4A5434CC6A62E050737157993, taskId=task_1463524593124_0001_1_00_000000, startTime=1463524593322, finishTime=1463524593372, timeTaken=50, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
], counters=Counters: 0
2016-05-17 22:36:33,373 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524593124_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:36:33,373 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:33,371 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=E7F6CAB4A5434CC6A62E050737157993, index=0
2016-05-17 22:36:33,371 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:33,377 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:33,376 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=E7F6CAB4A5434CC6A62E050737157993, sourceVertex=0DCCAEEABDBE43ECB61D39F1D021DF5A
2016-05-17 22:36:33,376 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993] : 1
2016-05-17 22:36:33,378 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993] because task failed: task_1463524593124_0001_1_00_000000
2016-05-17 22:36:33,378 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:36:33,378 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:36:33,378 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993]
2016-05-17 22:36:33,378 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=D7184FB845BB43FFAAB272F0AA9582FE, vertexId=vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993]
2016-05-17 22:36:33,382 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=E7F6CAB4A5434CC6A62E050737157993, sourceVertex=D7184FB845BB43FFAAB272F0AA9582FE
2016-05-17 22:36:33,382 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:33,383 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593124_0001_1][Event:VERTEX_FINISHED]: vertexName=E7F6CAB4A5434CC6A62E050737157993, vertexId=vertex_1463524593124_0001_1_00, initRequestedTime=1463524593298, initedTime=1463524593310, startRequestedTime=1463524593310, startedTime=1463524593310, finishTime=1463524593380, timeTaken=70, status=FAILED, diagnostics=Task failed, taskId=task_1463524593124_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524593322, firstTasksToStart=[ task_1463524593124_0001_1_00_000000 ], lastTaskFinishTime=1463524593371, lastTasksToFinish=[ task_1463524593124_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:36:33,383 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524593124_0001_00_000001
2016-05-17 22:36:33,387 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524593124_0001_00_000001
2016-05-17 22:36:33,388 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:36:33,388 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593124_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524593124_0001_00_000001, stoppedTime=1463524593388, exitStatus=0
2016-05-17 22:36:33,388 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:33,388 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524593124_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:33,389 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524593124_0001_1_00_000000_0 failed: true container: container_1463524593124_0001_00_000001 numFailedTAs: 0
2016-05-17 22:36:33,391 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:36:33,391 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:36:33,392 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524593124_0001_1
2016-05-17 22:36:33,392 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: D7184FB845BB43FFAAB272F0AA9582FE for vertex: vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993]
2016-05-17 22:36:33,391 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524593124_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:36:33,393 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524593124_0001_00_000001 completed but with errors
2016-05-17 22:36:33,393 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524593124_0001_1, queueSize=0
2016-05-17 22:36:33,394 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:33,394 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593124_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524593124_0001_1, startTime=1463524593297, finishTime=1463524593393, timeTaken=96, status=FAILED, diagnostics=Vertex failed, vertexName=E7F6CAB4A5434CC6A62E050737157993, vertexId=vertex_1463524593124_0001_1_00, diagnostics=[Task failed, taskId=task_1463524593124_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524593124_0001_1_00 [E7F6CAB4A5434CC6A62E050737157993] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:36:33,396 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524593124_0001_1 finished with state: FAILED
2016-05-17 22:36:33,396 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524593124_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:33,397 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524593124_0001_00_000001
2016-05-17 22:36:33,397 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524593124_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:36:33,397 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524593124_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:33 Completed Dag: dag_1463524593124_0001_1
2016-05-17 22:36:33,398 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524593124_0001_1, dagState=FAILED
2016-05-17 22:36:33,398 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524593124_0001_1_post
2016-05-17 22:36:33,398 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:33,398 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:33,399 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524593124_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:36:33,405 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/-1_VALUES_ALL, applicationId=application_1463524593124_0001
2016-05-17 22:36:33,407 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:33,407 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:33,407 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:33,407 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:33,407 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:33,408 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:33,408 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:33,409 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:33,409 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:33,409 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:33,409 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:33,410 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:33,410 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:33,414 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8C9887C9EC164FFFA1199073D6A50527/A7153FC73C874DDEB10F793D6D1FF3EF/.tez/application_1463524593124_0001
2016-05-17 22:36:33,414 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping all jobs
2016-05-17 22:36:33,414 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping: (1/1) ...blyplatform/-1_VALUES_ALL
2016-05-17 22:36:33,415 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopped all jobs
2016-05-17 22:36:33,415 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_ALL/_temporary
2016-05-17 22:36:33,416 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.311
2016-05-17 22:36:33,416 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_VALUES_ALL
2016-05-17 22:36:33,447 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:33,447 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:33,483 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 22:36:33,489 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-17 22:36:33,490 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:33,490 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:33,490 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:33,504 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:33,504 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_REPLACE
2016-05-17 22:36:33,516 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:33,516 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:33,516 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_REPLACE"]
2016-05-17 22:36:33,517 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:33,517 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:33,517 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:33,518 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...latform/-1_VALUES_REPLACE
2016-05-17 22:36:33,518 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:33,530 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:33,534 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:33,535 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:33,535 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:33,541 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:33,541 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:33,545 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/62417AFF863E4CD7B1A1B4A8B9482A2B/A1E54672D4CB45FF9F21D9B5455A6EA5/.tez/application_1463524593535_0001 doesn't exist and is created
2016-05-17 22:36:33,561 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:33,561 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:33,561 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/62417AFF863E4CD7B1A1B4A8B9482A2B/A1E54672D4CB45FF9F21D9B5455A6EA5/.tez/application_1463524593535_0001
2016-05-17 22:36:33,565 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524593535_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:33,575 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:33,576 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:33,577 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:33,577 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:33,578 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/62417AFF863E4CD7B1A1B4A8B9482A2B/A1E54672D4CB45FF9F21D9B5455A6EA5/.tez/application_1463524593535_0001/localmode-log-dir
2016-05-17 22:36:33,578 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/62417AFF863E4CD7B1A1B4A8B9482A2B/A1E54672D4CB45FF9F21D9B5455A6EA5/.tez/application_1463524593535_0001/localmode-log-dir/history.txt.appattempt_1463524593535_0001_000000, maxErrors=10
2016-05-17 22:36:33,578 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:33,579 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524593535_0001_000000, appSubmitTime=1463524593564, launchTime=1463524593565
2016-05-17 22:36:33,583 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:33,587 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49965
2016-05-17 22:36:33,587 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:33,590 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:33,590 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:33,591 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:33,593 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524593535_0001_000000, startTime=1463524593593
2016-05-17 22:36:33,593 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:33,593 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:33,661 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:33,661 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:33,661 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/-1_VALUES_REPLACE, applicationId=application_1463524593535_0001, dagName=[62417AFF863E4CD7B1A1B4A8B9482A2B/A1E54672D4CB45FF9F21D9B5455A6EA5] each/(1/1) ...latform/-1_VALUES_REPLACE
2016-05-17 22:36:33,678 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [62417AFF863E4CD7B1A1B4A8B9482A2B/A1E54672D4CB45FF9F21D9B5455A6EA5] each/(1/1) ...latform/-1_VALUES_REPLACE
2016-05-17 22:36:33,680 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524593535_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/62417AFF863E4CD7B1A1B4A8B9482A2B/A1E54672D4CB45FF9F21D9B5455A6EA5/.tez/application_1463524593535_0001/localmode-log-dir/dag_1463524593535_0001_1.dot
2016-05-17 22:36:33,680 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/62417AFF863E4CD7B1A1B4A8B9482A2B/A1E54672D4CB45FF9F21D9B5455A6EA5/.tez/application_1463524593535_0001/localmode-log-dir/dag_1463524593535_0001_1-tez-dag.pb.txt
2016-05-17 22:36:33,686 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524593535_0001_1
2016-05-17 22:36:33,686 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:33,686 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [62417AFF863E4CD7B1A1B4A8B9482A2B/A1E54672D4CB45FF9F21D9B5455A6EA5] each/(1/1) ...latform/-1_VALUES_REPLACE
2016-05-17 22:36:33 Running Dag: dag_1463524593535_0001_1
2016-05-17 22:36:33,699 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:33,699 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593535_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524593535_0001_1, submitTime=1463524593678
2016-05-17 22:36:33,700 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E11C5D11EA52434CBC20EC03D94005A2
2016-05-17 22:36:33,700 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:33,700 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:33,701 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593535_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524593535_0001_1, initTime=1463524593699
2016-05-17 22:36:33,701 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524593535_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:33,701 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593535_0001_1][Event:DAG_STARTED]: dagID=dag_1463524593535_0001_1, startTime=1463524593701
2016-05-17 22:36:33,701 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:33,701 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:33,702 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524593535_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:33,702 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E11C5D11EA52434CBC20EC03D94005A2 : {B4DBA9EFB6D946E5BFCD2834FF9F83FA={InputName=B4DBA9EFB6D946E5BFCD2834FF9F83FA}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:33,702 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: B4DBA9EFB6D946E5BFCD2834FF9F83FA, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:33,702 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2]
2016-05-17 22:36:33,702 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2]
2016-05-17 22:36:33,702 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2]
2016-05-17 22:36:33,703 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2]. Starting root input initializers: 1
2016-05-17 22:36:33,703 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/-1_VALUES_REPLACE, applicationId=application_1463524593535_0001, dagName=[62417AFF863E4CD7B1A1B4A8B9482A2B/A1E54672D4CB45FF9F21D9B5455A6EA5] each/(1/1) ...latform/-1_VALUES_REPLACE
2016-05-17 22:36:33,703 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:33,703 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: B4DBA9EFB6D946E5BFCD2834FF9F83FA on vertex vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2]
2016-05-17 22:36:33,706 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:33,706 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input B4DBA9EFB6D946E5BFCD2834FF9F83FA asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:33,707 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:33,707 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:33,709 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:33,709 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: B4DBA9EFB6D946E5BFCD2834FF9F83FA on vertex vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2]
2016-05-17 22:36:33,709 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:33,710 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {B4DBA9EFB6D946E5BFCD2834FF9F83FA=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:33,710 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2] parallelism set to 1
2016-05-17 22:36:33,710 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:33,711 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593535_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524593535_0001_1_00, eventCount=1
2016-05-17 22:36:33,711 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2]
2016-05-17 22:36:33,711 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=F4B137B1C576435A99A1699B53EBCEF2, vertexId=vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:33,711 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=F4B137B1C576435A99A1699B53EBCEF2, vertexId=vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2]
2016-05-17 22:36:33,713 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=F4B137B1C576435A99A1699B53EBCEF2, vertexName=E11C5D11EA52434CBC20EC03D94005A2, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:33,713 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=F4B137B1C576435A99A1699B53EBCEF2, vertexId=vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2]
2016-05-17 22:36:33,714 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593535_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E11C5D11EA52434CBC20EC03D94005A2, vertexId=vertex_1463524593535_0001_1_00, initRequestedTime=1463524593702, initedTime=1463524593714, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:33,714 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:33,714 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2] with distanceFromRoot: 0
2016-05-17 22:36:33,715 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:33,715 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E11C5D11EA52434CBC20EC03D94005A2
2016-05-17 22:36:33,715 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2]
2016-05-17 22:36:33,716 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593535_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524593535_0001_1_00, startRequestedTime=1463524593715, startedTime=1463524593715
2016-05-17 22:36:33,716 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:33,716 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:33,717 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:33,718 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593535_0001_1][Event:TASK_STARTED]: vertexName=E11C5D11EA52434CBC20EC03D94005A2, taskId=task_1463524593535_0001_1_00_000000, scheduledTime=1463524593717, launchTime=1463524593717
2016-05-17 22:36:33,718 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524593535_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:33,718 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2]
2016-05-17 22:36:33,718 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [62417AFF863E4CD7B1A1B4A8B9482A2B/A1E54672D4CB45FF9F21D9B5455A6EA5] each/(1/1) ...latform/-1_VALUES_REPLACE, VertexName: E11C5D11EA52434CBC20EC03D94005A2, VertexParallelism: 1, TaskAttemptID:attempt_1463524593535_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B4DBA9EFB6D946E5BFCD2834FF9F83FA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F4B137B1C576435A99A1699B53EBCEF2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:33,719 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524593535_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:33,719 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:33,720 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:33,720 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:33,721 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:33,721 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524593535_0001_00_000001
2016-05-17 22:36:33,721 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524593535_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:33,721 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524593535_0001, with dag id: dag_1463524593535_0001_1
2016-05-17 22:36:33,724 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524593535_0001_00_000001
2016-05-17 22:36:33,724 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:33,727 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:33,727 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524593535_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:33,727 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593535_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524593535_0001_00_000001, launchTime=1463524593726
2016-05-17 22:36:33,727 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524593535_0001_00_000001
2016-05-17 22:36:33,731 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524593535_0001_1_00_000000_0] to container: [container_1463524593535_0001_00_000001]
2016-05-17 22:36:33,731 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524593535_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:33,731 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524593535_0001_00_000001 given task: attempt_1463524593535_0001_1_00_000000_0
2016-05-17 22:36:33,732 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524593535_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524593535_0001_1_00_000000_0
2016-05-17 22:36:33 Starting to run new task attempt: attempt_1463524593535_0001_1_00_000000_0
2016-05-17 22:36:33,732 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524593535_0001_1_00_000000_0] started. Is using containerId: [container_1463524593535_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:33,732 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524593535_0001_1_00_000000_0
2016-05-17 22:36:33,732 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:33,732 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:33,732 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:33,732 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593535_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E11C5D11EA52434CBC20EC03D94005A2, taskAttemptId=attempt_1463524593535_0001_1_00_000000_0, startTime=1463524593732, containerId=container_1463524593535_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524593535_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:33,734 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524593535_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:33,735 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524593535_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:33,732 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:33,732 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:33,736 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:33,736 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:33,736 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [62417AFF863E4CD7B1A1B4A8B9482A2B/A1E54672D4CB45FF9F21D9B5455A6EA5] each/(1/1) ...latform/-1_VALUES_REPLACE, VertexName: E11C5D11EA52434CBC20EC03D94005A2, VertexParallelism: 1, TaskAttemptID:attempt_1463524593535_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B4DBA9EFB6D946E5BFCD2834FF9F83FA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F4B137B1C576435A99A1699B53EBCEF2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:33,737 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:33,737 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524593535_0001_1_00_000000_0
2016-05-17 22:36:33,737 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:33,738 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:33,738 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524593535_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:33,740 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B4DBA9EFB6D946E5BFCD2834FF9F83FA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:33,740 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F4B137B1C576435A99A1699B53EBCEF2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:33,740 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:33,740 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:33,740 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:33,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F4B137B1C576435A99A1699B53EBCEF2
2016-05-17 22:36:33,741 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:33,741 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: F4B137B1C576435A99A1699B53EBCEF2
2016-05-17 22:36:33,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B4DBA9EFB6D946E5BFCD2834FF9F83FA
2016-05-17 22:36:33,741 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: B4DBA9EFB6D946E5BFCD2834FF9F83FA
2016-05-17 22:36:33,745 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:33,746 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:33,746 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: B4DBA9EFB6D946E5BFCD2834FF9F83FA
2016-05-17 22:36:33,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B4DBA9EFB6D946E5BFCD2834FF9F83FA
2016-05-17 22:36:33,754 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E11C5D11EA52434CBC20EC03D94005A2, ordinal: 0
2016-05-17 22:36:33,754 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E11C5D11EA52434CBC20EC03D94005A2, mem on start (mb), free: 199, total: 388, max: 672
2016-05-17 22:36:33,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:33,755 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:33,755 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:33,759 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:33,759 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F4B137B1C576435A99A1699B53EBCEF2
2016-05-17 22:36:33,760 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:33,760 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:33,760 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:33,761 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:33,761 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:33,761 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:33,762 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:33,762 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:33,762 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F4B137B1C576435A99A1699B53EBCEF2, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:33,763 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B4DBA9EFB6D946E5BFCD2834FF9F83FA, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:33,763 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:33,764 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B4DBA9EFB6D946E5BFCD2834FF9F83FA
2016-05-17 22:36:33,764 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:33,764 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B4DBA9EFB6D946E5BFCD2834FF9F83FA
2016-05-17 22:36:33,765 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B4DBA9EFB6D946E5BFCD2834FF9F83FA being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:33,766 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:33,766 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:33,767 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:33,767 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524593535_0001_1_00_000000_0
2016-05-17 22:36:33,769 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:33,769 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2a147b53
2016-05-17 22:36:33,770 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:33,770 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: B4DBA9EFB6D946E5BFCD2834FF9F83FA
2016-05-17 22:36:33,771 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_REPLACE"], id: F4B137B1C576435A99A1699B53EBCEF2
2016-05-17 22:36:33,771 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_REPLACE"]
2016-05-17 22:36:33,771 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:33,771 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:33,772 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2a147b53
2016-05-17 22:36:33,772 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:33,772 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:33,772 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E11C5D11EA52434CBC20EC03D94005A2, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:33,773 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:33,773 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:33,774 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:33,774 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:33,775 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524593535_0001_1_00_000000_0
2016-05-17 22:36:33,775 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524593535_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:33,776 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524593535_0001_1_00_000000_0
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:33,783 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407371776, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524593772, Process_Duration=4, Process_End_Time=1463524593776, Read_Duration=1, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:36:33,783 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E11C5D11EA52434CBC20EC03D94005A2, mem on close (mb), free: 197, total: 388, max: 672
2016-05-17 22:36:33,783 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:33,783 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=E11C5D11EA52434CBC20EC03D94005A2, index=0
2016-05-17 22:36:33,783 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593535_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E11C5D11EA52434CBC20EC03D94005A2, taskAttemptId=attempt_1463524593535_0001_1_00_000000_0, startTime=1463524593732, finishTime=1463524593783, timeTaken=51, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407371776, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524593772, Process_Duration=4, Process_End_Time=1463524593776, Read_Duration=1, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:36:33,784 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524593535_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:36:33,784 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524593535_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:36:33,784 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:33,784 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593535_0001_1][Event:TASK_FINISHED]: vertexName=E11C5D11EA52434CBC20EC03D94005A2, taskId=task_1463524593535_0001_1_00_000000, startTime=1463524593732, finishTime=1463524593784, timeTaken=52, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
], counters=Counters: 0
2016-05-17 22:36:33,784 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524593535_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:36:33,785 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2] : 1
2016-05-17 22:36:33,785 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2] because task failed: task_1463524593535_0001_1_00_000000
2016-05-17 22:36:33,785 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:36:33,785 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:36:33,785 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2]
2016-05-17 22:36:33,785 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=F4B137B1C576435A99A1699B53EBCEF2, vertexId=vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2]
2016-05-17 22:36:33,787 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:33,787 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593535_0001_1][Event:VERTEX_FINISHED]: vertexName=E11C5D11EA52434CBC20EC03D94005A2, vertexId=vertex_1463524593535_0001_1_00, initRequestedTime=1463524593702, initedTime=1463524593714, startRequestedTime=1463524593715, startedTime=1463524593715, finishTime=1463524593786, timeTaken=71, status=FAILED, diagnostics=Task failed, taskId=task_1463524593535_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524593732, firstTasksToStart=[ task_1463524593535_0001_1_00_000000 ], lastTaskFinishTime=1463524593783, lastTasksToFinish=[ task_1463524593535_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:36:33,788 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:36:33,788 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:36:33,788 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:36:33,788 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524593535_0001_1
2016-05-17 22:36:33,788 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: F4B137B1C576435A99A1699B53EBCEF2 for vertex: vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2]
2016-05-17 22:36:33,787 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:33,788 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=E11C5D11EA52434CBC20EC03D94005A2, sourceVertex=B4DBA9EFB6D946E5BFCD2834FF9F83FA
2016-05-17 22:36:33,791 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=E11C5D11EA52434CBC20EC03D94005A2, sourceVertex=F4B137B1C576435A99A1699B53EBCEF2
2016-05-17 22:36:33,791 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524593535_0001_1, queueSize=0
2016-05-17 22:36:33,791 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524593535_0001_00_000001
2016-05-17 22:36:33,792 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524593535_0001_00_000001
2016-05-17 22:36:33,793 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:33,794 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593535_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524593535_0001_1, startTime=1463524593701, finishTime=1463524593790, timeTaken=89, status=FAILED, diagnostics=Vertex failed, vertexName=E11C5D11EA52434CBC20EC03D94005A2, vertexId=vertex_1463524593535_0001_1_00, diagnostics=[Task failed, taskId=task_1463524593535_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524593535_0001_1_00 [E11C5D11EA52434CBC20EC03D94005A2] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:36:33,794 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524593535_0001_1 finished with state: FAILED
2016-05-17 22:36:33,794 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524593535_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:36:33,794 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524593535_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:33,795 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524593535_0001_00_000001 completed but with errors
2016-05-17 22:36:33,795 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593535_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524593535_0001_00_000001, stoppedTime=1463524593795, exitStatus=0
2016-05-17 22:36:33,795 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:33,796 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524593535_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:33,796 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524593535_0001_1_00_000000_0 failed: true container: container_1463524593535_0001_00_000001 numFailedTAs: 0
2016-05-17 22:36:33 Completed Dag: dag_1463524593535_0001_1
2016-05-17 22:36:33,796 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524593535_0001_1, dagState=FAILED
2016-05-17 22:36:33,796 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524593535_0001_1_post
2016-05-17 22:36:33,796 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:33,796 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:33,797 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524593535_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:36:33,797 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524593535_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:33,797 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524593535_0001_00_000001
2016-05-17 22:36:33,797 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524593535_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:36:33,800 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/-1_VALUES_REPLACE, applicationId=application_1463524593535_0001
2016-05-17 22:36:33,802 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:33,802 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:33,802 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:33,803 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:33,803 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:33,803 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:33,808 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:33,808 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:33,809 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:33,809 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:33,809 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:33,810 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:33,810 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:33,815 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/62417AFF863E4CD7B1A1B4A8B9482A2B/A1E54672D4CB45FF9F21D9B5455A6EA5/.tez/application_1463524593535_0001
2016-05-17 22:36:33,815 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping all jobs
2016-05-17 22:36:33,815 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping: (1/1) ...latform/-1_VALUES_REPLACE
2016-05-17 22:36:33,816 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopped all jobs
2016-05-17 22:36:33,816 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_REPLACE/_temporary
2016-05-17 22:36:33,818 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.300
2016-05-17 22:36:33,818 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_VALUES_REPLACE
2016-05-17 22:36:33,853 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:33,853 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:33,902 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 22:36:33,912 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-05-17 22:36:33,913 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:33,913 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:33,913 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:33,935 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:33,936 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_SWAP
2016-05-17 22:36:33,950 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:33,951 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:33,951 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_SWAP"]
2016-05-17 22:36:33,951 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:33,951 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:33,952 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:33,953 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...lyplatform/-1_VALUES_SWAP
2016-05-17 22:36:33,953 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:33,971 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:33,976 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:33,977 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:33,977 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:33,986 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E4F4759D000447C18F20ECD4E48B4B9B/E84D65750ABF4BAE97CF8E6F1E24F8C5/.tez/application_1463524593977_0001 doesn't exist and is created
2016-05-17 22:36:34,006 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:34,006 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:34,007 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E4F4759D000447C18F20ECD4E48B4B9B/E84D65750ABF4BAE97CF8E6F1E24F8C5/.tez/application_1463524593977_0001
2016-05-17 22:36:34,010 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524593977_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:34,022 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:34,023 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:34,025 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:34,025 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:34,025 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E4F4759D000447C18F20ECD4E48B4B9B/E84D65750ABF4BAE97CF8E6F1E24F8C5/.tez/application_1463524593977_0001/localmode-log-dir
2016-05-17 22:36:34,025 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E4F4759D000447C18F20ECD4E48B4B9B/E84D65750ABF4BAE97CF8E6F1E24F8C5/.tez/application_1463524593977_0001/localmode-log-dir/history.txt.appattempt_1463524593977_0001_000000, maxErrors=10
2016-05-17 22:36:34,025 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:34,026 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524593977_0001_000000, appSubmitTime=1463524594010, launchTime=1463524594010
2016-05-17 22:36:34,032 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:34,033 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:34,034 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:34,035 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:34,037 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:58332
2016-05-17 22:36:34,038 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:34,040 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524593977_0001_000000, startTime=1463524594040
2016-05-17 22:36:34,040 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:34,041 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:34,107 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:34,107 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:34,107 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/-1_VALUES_SWAP, applicationId=application_1463524593977_0001, dagName=[E4F4759D000447C18F20ECD4E48B4B9B/E84D65750ABF4BAE97CF8E6F1E24F8C5] each/(1/1) ...lyplatform/-1_VALUES_SWAP
2016-05-17 22:36:34,125 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [E4F4759D000447C18F20ECD4E48B4B9B/E84D65750ABF4BAE97CF8E6F1E24F8C5] each/(1/1) ...lyplatform/-1_VALUES_SWAP
2016-05-17 22:36:34,127 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524593977_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E4F4759D000447C18F20ECD4E48B4B9B/E84D65750ABF4BAE97CF8E6F1E24F8C5/.tez/application_1463524593977_0001/localmode-log-dir/dag_1463524593977_0001_1.dot
2016-05-17 22:36:34,127 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E4F4759D000447C18F20ECD4E48B4B9B/E84D65750ABF4BAE97CF8E6F1E24F8C5/.tez/application_1463524593977_0001/localmode-log-dir/dag_1463524593977_0001_1-tez-dag.pb.txt
2016-05-17 22:36:34,133 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524593977_0001_1
2016-05-17 22:36:34,133 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:34,133 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [E4F4759D000447C18F20ECD4E48B4B9B/E84D65750ABF4BAE97CF8E6F1E24F8C5] each/(1/1) ...lyplatform/-1_VALUES_SWAP
2016-05-17 22:36:34 Running Dag: dag_1463524593977_0001_1
2016-05-17 22:36:34,146 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:34,146 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593977_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524593977_0001_1, submitTime=1463524594126
2016-05-17 22:36:34,146 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C83D36A828CE4CFAB75D0E2F2F236F22
2016-05-17 22:36:34,147 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:34,147 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:34,147 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593977_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524593977_0001_1, initTime=1463524594146
2016-05-17 22:36:34,147 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524593977_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:34,148 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593977_0001_1][Event:DAG_STARTED]: dagID=dag_1463524593977_0001_1, startTime=1463524594148
2016-05-17 22:36:34,148 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:34,148 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524593977_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:34,148 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C83D36A828CE4CFAB75D0E2F2F236F22 : {E2EE00E8BC2F49F0ACED9D3C05B1821B={InputName=E2EE00E8BC2F49F0ACED9D3C05B1821B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:34,148 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:34,148 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: E2EE00E8BC2F49F0ACED9D3C05B1821B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:34,149 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22]
2016-05-17 22:36:34,149 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22]
2016-05-17 22:36:34,149 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22]
2016-05-17 22:36:34,150 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22]. Starting root input initializers: 1
2016-05-17 22:36:34,150 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:34,150 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: E2EE00E8BC2F49F0ACED9D3C05B1821B on vertex vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22]
2016-05-17 22:36:34,150 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/-1_VALUES_SWAP, applicationId=application_1463524593977_0001, dagName=[E4F4759D000447C18F20ECD4E48B4B9B/E84D65750ABF4BAE97CF8E6F1E24F8C5] each/(1/1) ...lyplatform/-1_VALUES_SWAP
2016-05-17 22:36:34,152 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:34,152 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input E2EE00E8BC2F49F0ACED9D3C05B1821B asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:34,152 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:34,153 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:34,154 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:34,154 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: E2EE00E8BC2F49F0ACED9D3C05B1821B on vertex vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22]
2016-05-17 22:36:34,154 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:34,155 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {E2EE00E8BC2F49F0ACED9D3C05B1821B=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:34,155 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22] parallelism set to 1
2016-05-17 22:36:34,155 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:34,155 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593977_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524593977_0001_1_00, eventCount=1
2016-05-17 22:36:34,155 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22]
2016-05-17 22:36:34,155 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=52EC95714F204C01BB2CCA0EB2C726AE, vertexId=vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:34,155 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=52EC95714F204C01BB2CCA0EB2C726AE, vertexId=vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22]
2016-05-17 22:36:34,157 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=52EC95714F204C01BB2CCA0EB2C726AE, vertexName=C83D36A828CE4CFAB75D0E2F2F236F22, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:34,158 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=52EC95714F204C01BB2CCA0EB2C726AE, vertexId=vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22]
2016-05-17 22:36:34,159 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:34,159 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593977_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C83D36A828CE4CFAB75D0E2F2F236F22, vertexId=vertex_1463524593977_0001_1_00, initRequestedTime=1463524594148, initedTime=1463524594159, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:34,159 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22] with distanceFromRoot: 0
2016-05-17 22:36:34,159 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:34,159 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C83D36A828CE4CFAB75D0E2F2F236F22
2016-05-17 22:36:34,160 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22]
2016-05-17 22:36:34,160 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:34,160 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593977_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524593977_0001_1_00, startRequestedTime=1463524594159, startedTime=1463524594159
2016-05-17 22:36:34,160 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:34,160 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:34,161 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:34,161 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:34,161 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593977_0001_1][Event:TASK_STARTED]: vertexName=C83D36A828CE4CFAB75D0E2F2F236F22, taskId=task_1463524593977_0001_1_00_000000, scheduledTime=1463524594161, launchTime=1463524594161
2016-05-17 22:36:34,162 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524593977_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:34,162 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22]
2016-05-17 22:36:34,162 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E4F4759D000447C18F20ECD4E48B4B9B/E84D65750ABF4BAE97CF8E6F1E24F8C5] each/(1/1) ...lyplatform/-1_VALUES_SWAP, VertexName: C83D36A828CE4CFAB75D0E2F2F236F22, VertexParallelism: 1, TaskAttemptID:attempt_1463524593977_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E2EE00E8BC2F49F0ACED9D3C05B1821B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=52EC95714F204C01BB2CCA0EB2C726AE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:34,179 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524593977_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:34,179 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:34,180 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:34,181 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:34,181 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:34,181 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524593977_0001_00_000001
2016-05-17 22:36:34,181 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524593977_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:34,183 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524593977_0001_00_000001
2016-05-17 22:36:34,183 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:34,183 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524593977_0001, with dag id: dag_1463524593977_0001_1
2016-05-17 22:36:34,184 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524593977_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:34,184 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:34,185 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593977_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524593977_0001_00_000001, launchTime=1463524594184
2016-05-17 22:36:34,186 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524593977_0001_00_000001
2016-05-17 22:36:34,189 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524593977_0001_1_00_000000_0] to container: [container_1463524593977_0001_00_000001]
2016-05-17 22:36:34,189 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524593977_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:34,189 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524593977_0001_00_000001 given task: attempt_1463524593977_0001_1_00_000000_0
2016-05-17 22:36:34,189 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524593977_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524593977_0001_1_00_000000_0
2016-05-17 22:36:34,189 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524593977_0001_1_00_000000_0] started. Is using containerId: [container_1463524593977_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:34,189 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593977_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C83D36A828CE4CFAB75D0E2F2F236F22, taskAttemptId=attempt_1463524593977_0001_1_00_000000_0, startTime=1463524594189, containerId=container_1463524593977_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524593977_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:34 Starting to run new task attempt: attempt_1463524593977_0001_1_00_000000_0
2016-05-17 22:36:34,190 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524593977_0001_1_00_000000_0
2016-05-17 22:36:34,190 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524593977_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:34,190 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:34,190 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524593977_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:34,190 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:34,190 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:34,190 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:34,190 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:34,191 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:34,191 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:34,191 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E4F4759D000447C18F20ECD4E48B4B9B/E84D65750ABF4BAE97CF8E6F1E24F8C5] each/(1/1) ...lyplatform/-1_VALUES_SWAP, VertexName: C83D36A828CE4CFAB75D0E2F2F236F22, VertexParallelism: 1, TaskAttemptID:attempt_1463524593977_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E2EE00E8BC2F49F0ACED9D3C05B1821B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=52EC95714F204C01BB2CCA0EB2C726AE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:34,191 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:34,192 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524593977_0001_1_00_000000_0
2016-05-17 22:36:34,192 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:34,192 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:34,193 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524593977_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:34,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E2EE00E8BC2F49F0ACED9D3C05B1821B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:34,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:34,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:34,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E2EE00E8BC2F49F0ACED9D3C05B1821B
2016-05-17 22:36:34,195 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: E2EE00E8BC2F49F0ACED9D3C05B1821B
2016-05-17 22:36:34,195 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=52EC95714F204C01BB2CCA0EB2C726AE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:34,196 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:34,196 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 52EC95714F204C01BB2CCA0EB2C726AE
2016-05-17 22:36:34,196 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:34,196 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 52EC95714F204C01BB2CCA0EB2C726AE
2016-05-17 22:36:34,198 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:34,198 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: E2EE00E8BC2F49F0ACED9D3C05B1821B
2016-05-17 22:36:34,198 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E2EE00E8BC2F49F0ACED9D3C05B1821B
2016-05-17 22:36:34,199 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:34,208 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:34,208 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 52EC95714F204C01BB2CCA0EB2C726AE
2016-05-17 22:36:34,208 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C83D36A828CE4CFAB75D0E2F2F236F22, ordinal: 0
2016-05-17 22:36:34,208 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C83D36A828CE4CFAB75D0E2F2F236F22, mem on start (mb), free: 341, total: 387, max: 672
2016-05-17 22:36:34,208 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:34,208 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:34,208 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:34,208 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:34,209 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:34,209 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:34,209 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:34,209 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:34,209 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:34,209 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:34,209 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:34,209 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E2EE00E8BC2F49F0ACED9D3C05B1821B, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:34,210 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 52EC95714F204C01BB2CCA0EB2C726AE, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:34,210 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:34,210 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E2EE00E8BC2F49F0ACED9D3C05B1821B being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:34,210 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E2EE00E8BC2F49F0ACED9D3C05B1821B
2016-05-17 22:36:34,211 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:34,211 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E2EE00E8BC2F49F0ACED9D3C05B1821B
2016-05-17 22:36:34,211 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:34,211 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:34,212 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:34,212 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524593977_0001_1_00_000000_0
2016-05-17 22:36:34,213 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:34,214 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@280290fe
2016-05-17 22:36:34,215 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:34,216 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: E2EE00E8BC2F49F0ACED9D3C05B1821B
2016-05-17 22:36:34,216 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_SWAP"], id: 52EC95714F204C01BB2CCA0EB2C726AE
2016-05-17 22:36:34,216 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_SWAP"]
2016-05-17 22:36:34,216 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@280290fe
2016-05-17 22:36:34,217 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:34,217 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:34,217 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:34,217 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:34,218 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C83D36A828CE4CFAB75D0E2F2F236F22, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:34,219 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:34,222 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:34,224 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:34,224 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:34,224 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524593977_0001_1_00_000000_0
2016-05-17 22:36:34,224 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524593977_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:34,227 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524593977_0001_1_00_000000_0
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:34,229 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406323200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524594218, Process_Duration=9, Process_End_Time=1463524594227, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:36:34,229 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:34,229 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593977_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C83D36A828CE4CFAB75D0E2F2F236F22, taskAttemptId=attempt_1463524593977_0001_1_00_000000_0, startTime=1463524594189, finishTime=1463524594229, timeTaken=40, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406323200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524594218, Process_Duration=9, Process_End_Time=1463524594227, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:36:34,229 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C83D36A828CE4CFAB75D0E2F2F236F22, mem on close (mb), free: 339, total: 387, max: 672
2016-05-17 22:36:34,229 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524593977_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:36:34,229 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=C83D36A828CE4CFAB75D0E2F2F236F22, index=0
2016-05-17 22:36:34,230 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524593977_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:36:34,230 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593977_0001_1][Event:TASK_FINISHED]: vertexName=C83D36A828CE4CFAB75D0E2F2F236F22, taskId=task_1463524593977_0001_1_00_000000, startTime=1463524594189, finishTime=1463524594230, timeTaken=41, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
], counters=Counters: 0
2016-05-17 22:36:34,230 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524593977_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:36:34,231 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22] : 1
2016-05-17 22:36:34,231 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22] because task failed: task_1463524593977_0001_1_00_000000
2016-05-17 22:36:34,230 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=C83D36A828CE4CFAB75D0E2F2F236F22, sourceVertex=E2EE00E8BC2F49F0ACED9D3C05B1821B
2016-05-17 22:36:34,231 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:34,231 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:36:34,232 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:36:34,232 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22]
2016-05-17 22:36:34,233 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=52EC95714F204C01BB2CCA0EB2C726AE, vertexId=vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22]
2016-05-17 22:36:34,232 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:34,231 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=C83D36A828CE4CFAB75D0E2F2F236F22, sourceVertex=52EC95714F204C01BB2CCA0EB2C726AE
2016-05-17 22:36:34,235 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593977_0001_1][Event:VERTEX_FINISHED]: vertexName=C83D36A828CE4CFAB75D0E2F2F236F22, vertexId=vertex_1463524593977_0001_1_00, initRequestedTime=1463524594148, initedTime=1463524594159, startRequestedTime=1463524594159, startedTime=1463524594159, finishTime=1463524594234, timeTaken=75, status=FAILED, diagnostics=Task failed, taskId=task_1463524593977_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524594189, firstTasksToStart=[ task_1463524593977_0001_1_00_000000 ], lastTaskFinishTime=1463524594229, lastTasksToFinish=[ task_1463524593977_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:36:34,235 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:34,235 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524593977_0001_00_000001
2016-05-17 22:36:34,236 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524593977_0001_00_000001
2016-05-17 22:36:34,238 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524593977_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:36:34,238 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524593977_0001_00_000001 completed but with errors
2016-05-17 22:36:34,244 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:36:34,245 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593977_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524593977_0001_00_000001, stoppedTime=1463524594245, exitStatus=0
2016-05-17 22:36:34,245 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:34,246 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524593977_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:34,246 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524593977_0001_1_00_000000_0 failed: true container: container_1463524593977_0001_00_000001 numFailedTAs: 0
2016-05-17 22:36:34,246 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:36:34,246 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:36:34,246 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524593977_0001_1
2016-05-17 22:36:34,246 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 52EC95714F204C01BB2CCA0EB2C726AE for vertex: vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22]
2016-05-17 22:36:34,247 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524593977_0001_1, queueSize=0
2016-05-17 22:36:34,247 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:34,247 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524593977_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524593977_0001_1, startTime=1463524594148, finishTime=1463524594246, timeTaken=98, status=FAILED, diagnostics=Vertex failed, vertexName=C83D36A828CE4CFAB75D0E2F2F236F22, vertexId=vertex_1463524593977_0001_1_00, diagnostics=[Task failed, taskId=task_1463524593977_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524593977_0001_1_00 [C83D36A828CE4CFAB75D0E2F2F236F22] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:36:34,247 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524593977_0001_1 finished with state: FAILED
2016-05-17 22:36:34,247 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524593977_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:34,248 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524593977_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:36:34,248 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524593977_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:34,248 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524593977_0001_00_000001
2016-05-17 22:36:34 Completed Dag: dag_1463524593977_0001_1
2016-05-17 22:36:34,248 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524593977_0001_1, dagState=FAILED
2016-05-17 22:36:34,248 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524593977_0001_1_post
2016-05-17 22:36:34,248 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:34,248 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:34,248 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524593977_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:36:34,261 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/-1_VALUES_SWAP, applicationId=application_1463524593977_0001
2016-05-17 22:36:34,263 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:34,263 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:34,263 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:34,263 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:34,263 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:34,264 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:34,264 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:34,265 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:34,266 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:34,266 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:34,266 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:34,267 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:34,267 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:34,273 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E4F4759D000447C18F20ECD4E48B4B9B/E84D65750ABF4BAE97CF8E6F1E24F8C5/.tez/application_1463524593977_0001
2016-05-17 22:36:34,274 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping all jobs
2016-05-17 22:36:34,274 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping: (1/1) ...lyplatform/-1_VALUES_SWAP
2016-05-17 22:36:34,274 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopped all jobs
2016-05-17 22:36:34,275 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_VALUES_SWAP/_temporary
2016-05-17 22:36:34,275 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.325
2016-05-17 22:36:34,276 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_VALUES_SWAP
2016-05-17 22:36:34,298 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:34,299 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:34,330 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-05-17 22:36:34,337 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-17 22:36:34,338 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:34,339 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:34,339 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:34,353 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:34,354 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_ARGS_-1
2016-05-17 22:36:34,363 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:34,363 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:34,364 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_ARGS_-1"]
2016-05-17 22:36:34,364 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:34,364 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:34,364 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:34,365 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...semblyplatform/-1_ARGS_-1
2016-05-17 22:36:34,365 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:34,377 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:34,381 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:34,382 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:34,382 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:34,391 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2357AA1912FB450D84890CFDF78BDBEC/EB2319EE9CB1401A837810948C8AA33C/.tez/application_1463524594382_0001 doesn't exist and is created
2016-05-17 22:36:34,407 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:34,408 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:34,408 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2357AA1912FB450D84890CFDF78BDBEC/EB2319EE9CB1401A837810948C8AA33C/.tez/application_1463524594382_0001
2016-05-17 22:36:34,412 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524594382_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:34,422 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:34,423 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:34,425 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:34,425 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:34,425 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2357AA1912FB450D84890CFDF78BDBEC/EB2319EE9CB1401A837810948C8AA33C/.tez/application_1463524594382_0001/localmode-log-dir
2016-05-17 22:36:34,426 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2357AA1912FB450D84890CFDF78BDBEC/EB2319EE9CB1401A837810948C8AA33C/.tez/application_1463524594382_0001/localmode-log-dir/history.txt.appattempt_1463524594382_0001_000000, maxErrors=10
2016-05-17 22:36:34,427 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:34,427 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524594382_0001_000000, appSubmitTime=1463524594411, launchTime=1463524594411
2016-05-17 22:36:34,434 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:34,436 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:33521
2016-05-17 22:36:34,436 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:34,438 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:34,439 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:34,439 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:34,442 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524594382_0001_000000, startTime=1463524594442
2016-05-17 22:36:34,442 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:34,442 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:34,508 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:34,508 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:34,508 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...semblyplatform/-1_ARGS_-1, applicationId=application_1463524594382_0001, dagName=[2357AA1912FB450D84890CFDF78BDBEC/EB2319EE9CB1401A837810948C8AA33C] each/(1/1) ...semblyplatform/-1_ARGS_-1
2016-05-17 22:36:34,523 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [2357AA1912FB450D84890CFDF78BDBEC/EB2319EE9CB1401A837810948C8AA33C] each/(1/1) ...semblyplatform/-1_ARGS_-1
2016-05-17 22:36:34,525 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524594382_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2357AA1912FB450D84890CFDF78BDBEC/EB2319EE9CB1401A837810948C8AA33C/.tez/application_1463524594382_0001/localmode-log-dir/dag_1463524594382_0001_1.dot
2016-05-17 22:36:34,525 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2357AA1912FB450D84890CFDF78BDBEC/EB2319EE9CB1401A837810948C8AA33C/.tez/application_1463524594382_0001/localmode-log-dir/dag_1463524594382_0001_1-tez-dag.pb.txt
2016-05-17 22:36:34,540 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524594382_0001_1
2016-05-17 22:36:34,540 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:34,541 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [2357AA1912FB450D84890CFDF78BDBEC/EB2319EE9CB1401A837810948C8AA33C] each/(1/1) ...semblyplatform/-1_ARGS_-1
2016-05-17 22:36:34 Running Dag: dag_1463524594382_0001_1
2016-05-17 22:36:34,552 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594382_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524594382_0001_1, submitTime=1463524594524
2016-05-17 22:36:34,552 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:34,553 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D497CD6498A4447A9768FA81606BCB90
2016-05-17 22:36:34,553 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:34,554 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594382_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524594382_0001_1, initTime=1463524594552
2016-05-17 22:36:34,554 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:34,554 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524594382_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:34,555 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594382_0001_1][Event:DAG_STARTED]: dagID=dag_1463524594382_0001_1, startTime=1463524594555
2016-05-17 22:36:34,555 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:34,555 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:34,556 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524594382_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:34,556 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...semblyplatform/-1_ARGS_-1, applicationId=application_1463524594382_0001, dagName=[2357AA1912FB450D84890CFDF78BDBEC/EB2319EE9CB1401A837810948C8AA33C] each/(1/1) ...semblyplatform/-1_ARGS_-1
2016-05-17 22:36:34,556 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D497CD6498A4447A9768FA81606BCB90 : {701E7593F6F54B219A4A6CD76AA302D8={InputName=701E7593F6F54B219A4A6CD76AA302D8}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:34,558 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 701E7593F6F54B219A4A6CD76AA302D8, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:34,558 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90]
2016-05-17 22:36:34,558 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90]
2016-05-17 22:36:34,558 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90]
2016-05-17 22:36:34,558 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90]. Starting root input initializers: 1
2016-05-17 22:36:34,559 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:34,559 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 701E7593F6F54B219A4A6CD76AA302D8 on vertex vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90]
2016-05-17 22:36:34,561 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:34,561 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 701E7593F6F54B219A4A6CD76AA302D8 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:34,561 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:34,562 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:34,563 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:34,563 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 701E7593F6F54B219A4A6CD76AA302D8 on vertex vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90]
2016-05-17 22:36:34,563 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:34,563 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {701E7593F6F54B219A4A6CD76AA302D8=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:34,564 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90] parallelism set to 1
2016-05-17 22:36:34,564 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:34,564 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594382_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524594382_0001_1_00, eventCount=1
2016-05-17 22:36:34,564 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90]
2016-05-17 22:36:34,564 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=5E8565B896994780867FAEF4843611B5, vertexId=vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:34,565 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=5E8565B896994780867FAEF4843611B5, vertexId=vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90]
2016-05-17 22:36:34,566 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=5E8565B896994780867FAEF4843611B5, vertexName=D497CD6498A4447A9768FA81606BCB90, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:34,567 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=5E8565B896994780867FAEF4843611B5, vertexId=vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90]
2016-05-17 22:36:34,567 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594382_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D497CD6498A4447A9768FA81606BCB90, vertexId=vertex_1463524594382_0001_1_00, initRequestedTime=1463524594556, initedTime=1463524594567, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:34,568 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90] with distanceFromRoot: 0
2016-05-17 22:36:34,568 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:34,568 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:34,568 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D497CD6498A4447A9768FA81606BCB90
2016-05-17 22:36:34,569 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90]
2016-05-17 22:36:34,569 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:34,569 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594382_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524594382_0001_1_00, startRequestedTime=1463524594568, startedTime=1463524594568
2016-05-17 22:36:34,569 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:34,570 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:34,570 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594382_0001_1][Event:TASK_STARTED]: vertexName=D497CD6498A4447A9768FA81606BCB90, taskId=task_1463524594382_0001_1_00_000000, scheduledTime=1463524594570, launchTime=1463524594570
2016-05-17 22:36:34,570 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524594382_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:34,570 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90]
2016-05-17 22:36:34,571 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2357AA1912FB450D84890CFDF78BDBEC/EB2319EE9CB1401A837810948C8AA33C] each/(1/1) ...semblyplatform/-1_ARGS_-1, VertexName: D497CD6498A4447A9768FA81606BCB90, VertexParallelism: 1, TaskAttemptID:attempt_1463524594382_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=701E7593F6F54B219A4A6CD76AA302D8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=5E8565B896994780867FAEF4843611B5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:34,571 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524594382_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:34,571 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:34,572 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:34,573 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524594382_0001, with dag id: dag_1463524594382_0001_1
2016-05-17 22:36:34,575 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:34,575 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:34,575 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524594382_0001_00_000001
2016-05-17 22:36:34,576 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524594382_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:34,577 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524594382_0001_00_000001
2016-05-17 22:36:34,578 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:34,578 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524594382_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:34,579 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:34,579 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594382_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524594382_0001_00_000001, launchTime=1463524594578
2016-05-17 22:36:34,580 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524594382_0001_00_000001
2016-05-17 22:36:34,580 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524594382_0001_1_00_000000_0] to container: [container_1463524594382_0001_00_000001]
2016-05-17 22:36:34,581 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524594382_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:34,581 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524594382_0001_1_00_000000_0] started. Is using containerId: [container_1463524594382_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:34,581 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594382_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D497CD6498A4447A9768FA81606BCB90, taskAttemptId=attempt_1463524594382_0001_1_00_000000_0, startTime=1463524594581, containerId=container_1463524594382_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524594382_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:34,581 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:34,581 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524594382_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:34,581 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524594382_0001_00_000001 given task: attempt_1463524594382_0001_1_00_000000_0
2016-05-17 22:36:34,582 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524594382_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:34,582 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524594382_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524594382_0001_1_00_000000_0
2016-05-17 22:36:34 Starting to run new task attempt: attempt_1463524594382_0001_1_00_000000_0
2016-05-17 22:36:34,583 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524594382_0001_1_00_000000_0
2016-05-17 22:36:34,583 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:34,583 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:34,583 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:34,583 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:34,583 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:34,583 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:34,584 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2357AA1912FB450D84890CFDF78BDBEC/EB2319EE9CB1401A837810948C8AA33C] each/(1/1) ...semblyplatform/-1_ARGS_-1, VertexName: D497CD6498A4447A9768FA81606BCB90, VertexParallelism: 1, TaskAttemptID:attempt_1463524594382_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=701E7593F6F54B219A4A6CD76AA302D8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=5E8565B896994780867FAEF4843611B5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:34,584 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:34,584 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524594382_0001_1_00_000000_0
2016-05-17 22:36:34,584 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:34,585 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:34,586 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=701E7593F6F54B219A4A6CD76AA302D8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:34,587 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:34,587 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 701E7593F6F54B219A4A6CD76AA302D8
2016-05-17 22:36:34,587 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 701E7593F6F54B219A4A6CD76AA302D8
2016-05-17 22:36:34,588 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:34,589 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=5E8565B896994780867FAEF4843611B5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:34,590 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:34,590 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524594382_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:34,590 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:34,591 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 701E7593F6F54B219A4A6CD76AA302D8
2016-05-17 22:36:34,591 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 701E7593F6F54B219A4A6CD76AA302D8
2016-05-17 22:36:34,593 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 5E8565B896994780867FAEF4843611B5
2016-05-17 22:36:34,593 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:34,593 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 5E8565B896994780867FAEF4843611B5
2016-05-17 22:36:34,594 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:34,602 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D497CD6498A4447A9768FA81606BCB90, ordinal: 0
2016-05-17 22:36:34,603 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D497CD6498A4447A9768FA81606BCB90, mem on start (mb), free: 278, total: 387, max: 672
2016-05-17 22:36:34,603 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:34,603 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:34,603 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:34,607 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:34,607 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 5E8565B896994780867FAEF4843611B5
2016-05-17 22:36:34,607 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:34,607 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:34,608 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:34,609 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:34,609 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:34,609 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:34,609 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:34,609 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:34,609 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 701E7593F6F54B219A4A6CD76AA302D8, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:34,609 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 5E8565B896994780867FAEF4843611B5, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:34,609 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:34,609 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 701E7593F6F54B219A4A6CD76AA302D8 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:34,609 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:34,610 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:34,610 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 701E7593F6F54B219A4A6CD76AA302D8
2016-05-17 22:36:34,611 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:34,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 701E7593F6F54B219A4A6CD76AA302D8
2016-05-17 22:36:34,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:34,612 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524594382_0001_1_00_000000_0
2016-05-17 22:36:34,612 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:34,614 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@57f99115
2016-05-17 22:36:34,616 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@57f99115
2016-05-17 22:36:34,616 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:34,616 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:34,616 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:34,617 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 701E7593F6F54B219A4A6CD76AA302D8
2016-05-17 22:36:34,617 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_ARGS_-1"], id: 5E8565B896994780867FAEF4843611B5
2016-05-17 22:36:34,618 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_ARGS_-1"]
2016-05-17 22:36:34,618 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:34,618 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:34,619 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D497CD6498A4447A9768FA81606BCB90, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:34,621 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:34,622 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:34,622 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524594382_0001_1_00_000000_0
2016-05-17 22:36:34,623 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524594382_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:34,623 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524594382_0001_1_00_000000_0
2016-05-17 22:36:34,624 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D497CD6498A4447A9768FA81606BCB90, mem on close (mb), free: 277, total: 387, max: 672
2016-05-17 22:36:34 Completed running task attempt: attempt_1463524594382_0001_1_00_000000_0
2016-05-17 22:36:34,625 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524594382_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:34,626 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406323200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524594619, Process_Duration=4, Process_End_Time=1463524594623, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:34,626 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594382_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D497CD6498A4447A9768FA81606BCB90, taskAttemptId=attempt_1463524594382_0001_1_00_000000_0, startTime=1463524594581, finishTime=1463524594626, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406323200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524594619, Process_Duration=4, Process_End_Time=1463524594623, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:34,627 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:34,627 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524594382_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:34,627 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524594382_0001_1_00_000000_0
2016-05-17 22:36:34,627 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:34,626 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:34,628 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594382_0001_1][Event:TASK_FINISHED]: vertexName=D497CD6498A4447A9768FA81606BCB90, taskId=task_1463524594382_0001_1_00_000000, startTime=1463524594581, finishTime=1463524594627, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1463524594382_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406323200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524594619, Process_Duration=4, Process_End_Time=1463524594623, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:34,628 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524594382_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:34,628 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:34,628 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90] : 1
2016-05-17 22:36:34,628 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90]
2016-05-17 22:36:34,628 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:34,629 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524594382_0001_00_000001
2016-05-17 22:36:34,630 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:34,631 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594382_0001_1][Event:VERTEX_FINISHED]: vertexName=D497CD6498A4447A9768FA81606BCB90, vertexId=vertex_1463524594382_0001_1_00, initRequestedTime=1463524594556, initedTime=1463524594567, startRequestedTime=1463524594568, startedTime=1463524594568, finishTime=1463524594628, timeTaken=60, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406323200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524594619, Process_Duration=4, Process_End_Time=1463524594623, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524594581, firstTasksToStart=[ task_1463524594382_0001_1_00_000000 ], lastTaskFinishTime=1463524594626, lastTasksToFinish=[ task_1463524594382_0001_1_00_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524594382_0001_1_00_000000 ], longestDurationTasks=[ task_1463524594382_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:34,631 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:34,631 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524594382_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:34,631 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594382_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524594382_0001_00_000001, stoppedTime=1463524594631, exitStatus=0
2016-05-17 22:36:34,631 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524594382_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:34,631 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:34,632 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:34,632 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524594382_0001_1
2016-05-17 22:36:34,632 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594382_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524594382_0001_1
2016-05-17 22:36:34,632 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 5E8565B896994780867FAEF4843611B5 for vertex: vertex_1463524594382_0001_1_00 [D497CD6498A4447A9768FA81606BCB90]
2016-05-17 22:36:34,639 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524594382_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:34,639 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524594382_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:34,639 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524594382_0001_00_000001, Exiting
2016-05-17 22:36:34,639 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524594382_0001_00_000001
2016-05-17 22:36:34,639 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524594382_0001_00_000001
2016-05-17 22:36:34,640 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524594382_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:34,641 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524594382_0001_00_000001 completed successfully
2016-05-17 22:36:34,641 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524594382_0001_1, queueSize=0
2016-05-17 22:36:34,641 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:34,641 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594382_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524594382_0001_1, startTime=1463524594555, finishTime=1463524594639, timeTaken=84, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406323200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524594619, Process_Duration=4, Process_End_Time=1463524594623, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:34,641 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524594382_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:34,641 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524594382_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:34,642 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524594382_0001_00_000001
2016-05-17 22:36:34,643 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524594382_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:34 Completed Dag: dag_1463524594382_0001_1
2016-05-17 22:36:34,644 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524594382_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:34,644 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524594382_0001_1_post
2016-05-17 22:36:34,644 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:34,644 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:34,648 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...semblyplatform/-1_ARGS_-1, applicationId=application_1463524594382_0001
2016-05-17 22:36:34,650 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:34,650 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:34,650 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:34,650 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:34,650 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:34,651 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:34,651 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:34,651 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:34,652 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:34,652 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:34,652 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:34,652 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:34,653 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:34,657 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2357AA1912FB450D84890CFDF78BDBEC/EB2319EE9CB1401A837810948C8AA33C/.tez/application_1463524594382_0001
2016-05-17 22:36:34,657 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_ARGS_-1/_temporary
2016-05-17 22:36:34,658 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.294
2016-05-17 22:36:34,724 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_ARGS_-1
2016-05-17 22:36:34,745 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:34,746 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:34,775 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-05-17 22:36:34,776 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.006
2016-05-17 22:36:34,776 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:34,776 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:34,777 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_ARGS_'field'
2016-05-17 22:36:34,798 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:34,799 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:34,835 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 22:36:34,836 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:34,836 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:34,840 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-17 22:36:34,841 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:34,841 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:34,841 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:34,855 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:34,855 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_ARGS_RESULTS
2016-05-17 22:36:34,864 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:34,864 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:34,864 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_ARGS_RESULTS"]
2016-05-17 22:36:34,864 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:34,865 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:34,865 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:34,866 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...yplatform/-1_ARGS_RESULTS
2016-05-17 22:36:34,867 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:34,879 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:34,884 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:34,884 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:34,884 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:34,893 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/FA50ECBFE3414A188A739104E5AABC1C/ECFFCADB77994AA3A32078C9BD470F14/.tez/application_1463524594884_0001 doesn't exist and is created
2016-05-17 22:36:34,911 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:34,911 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:34,912 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/FA50ECBFE3414A188A739104E5AABC1C/ECFFCADB77994AA3A32078C9BD470F14/.tez/application_1463524594884_0001
2016-05-17 22:36:34,917 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524594884_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:34,928 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:34,929 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:34,938 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:34,938 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:34,938 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/FA50ECBFE3414A188A739104E5AABC1C/ECFFCADB77994AA3A32078C9BD470F14/.tez/application_1463524594884_0001/localmode-log-dir
2016-05-17 22:36:34,939 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/FA50ECBFE3414A188A739104E5AABC1C/ECFFCADB77994AA3A32078C9BD470F14/.tez/application_1463524594884_0001/localmode-log-dir/history.txt.appattempt_1463524594884_0001_000000, maxErrors=10
2016-05-17 22:36:34,939 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:34,939 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524594884_0001_000000, appSubmitTime=1463524594917, launchTime=1463524594917
2016-05-17 22:36:34,943 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:34,944 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:34,946 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:34,946 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:34,948 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:44455
2016-05-17 22:36:34,950 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:34,952 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524594884_0001_000000, startTime=1463524594952
2016-05-17 22:36:34,952 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:34,952 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:35,012 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:35,012 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:35,012 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/-1_ARGS_RESULTS, applicationId=application_1463524594884_0001, dagName=[FA50ECBFE3414A188A739104E5AABC1C/ECFFCADB77994AA3A32078C9BD470F14] each/(1/1) ...yplatform/-1_ARGS_RESULTS
2016-05-17 22:36:35,028 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [FA50ECBFE3414A188A739104E5AABC1C/ECFFCADB77994AA3A32078C9BD470F14] each/(1/1) ...yplatform/-1_ARGS_RESULTS
2016-05-17 22:36:35,030 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524594884_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/FA50ECBFE3414A188A739104E5AABC1C/ECFFCADB77994AA3A32078C9BD470F14/.tez/application_1463524594884_0001/localmode-log-dir/dag_1463524594884_0001_1.dot
2016-05-17 22:36:35,030 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/FA50ECBFE3414A188A739104E5AABC1C/ECFFCADB77994AA3A32078C9BD470F14/.tez/application_1463524594884_0001/localmode-log-dir/dag_1463524594884_0001_1-tez-dag.pb.txt
2016-05-17 22:36:35,038 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524594884_0001_1
2016-05-17 22:36:35,038 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:35,038 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [FA50ECBFE3414A188A739104E5AABC1C/ECFFCADB77994AA3A32078C9BD470F14] each/(1/1) ...yplatform/-1_ARGS_RESULTS
2016-05-17 22:36:35 Running Dag: dag_1463524594884_0001_1
2016-05-17 22:36:35,054 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:35,055 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594884_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524594884_0001_1, submitTime=1463524595028
2016-05-17 22:36:35,055 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BE38BD61DE67449CB4516A3E9B90B5BB
2016-05-17 22:36:35,056 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:35,057 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:35,057 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594884_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524594884_0001_1, initTime=1463524595055
2016-05-17 22:36:35,058 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524594884_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:35,061 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594884_0001_1][Event:DAG_STARTED]: dagID=dag_1463524594884_0001_1, startTime=1463524595058
2016-05-17 22:36:35,061 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:35,061 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524594884_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:35,061 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/-1_ARGS_RESULTS, applicationId=application_1463524594884_0001, dagName=[FA50ECBFE3414A188A739104E5AABC1C/ECFFCADB77994AA3A32078C9BD470F14] each/(1/1) ...yplatform/-1_ARGS_RESULTS
2016-05-17 22:36:35,061 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:35,061 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BE38BD61DE67449CB4516A3E9B90B5BB : {0C63A9F508D04B0583572246464AF2A7={InputName=0C63A9F508D04B0583572246464AF2A7}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:35,062 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 0C63A9F508D04B0583572246464AF2A7, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:35,062 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB]
2016-05-17 22:36:35,062 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB]
2016-05-17 22:36:35,062 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB]
2016-05-17 22:36:35,062 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB]. Starting root input initializers: 1
2016-05-17 22:36:35,063 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:35,063 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 0C63A9F508D04B0583572246464AF2A7 on vertex vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB]
2016-05-17 22:36:35,067 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:35,069 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 0C63A9F508D04B0583572246464AF2A7 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:35,069 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:35,070 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:35,071 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:35,072 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 0C63A9F508D04B0583572246464AF2A7 on vertex vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB]
2016-05-17 22:36:35,072 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:35,072 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {0C63A9F508D04B0583572246464AF2A7=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:35,073 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB] parallelism set to 1
2016-05-17 22:36:35,073 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:35,073 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594884_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524594884_0001_1_00, eventCount=1
2016-05-17 22:36:35,073 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB]
2016-05-17 22:36:35,073 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=1F2779593D19445EACAA3CB4E43715D7, vertexId=vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:35,074 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=1F2779593D19445EACAA3CB4E43715D7, vertexId=vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB]
2016-05-17 22:36:35,076 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=1F2779593D19445EACAA3CB4E43715D7, vertexName=BE38BD61DE67449CB4516A3E9B90B5BB, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:35,076 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=1F2779593D19445EACAA3CB4E43715D7, vertexId=vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB]
2016-05-17 22:36:35,078 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594884_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BE38BD61DE67449CB4516A3E9B90B5BB, vertexId=vertex_1463524594884_0001_1_00, initRequestedTime=1463524595061, initedTime=1463524595077, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:35,078 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:35,079 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB] with distanceFromRoot: 0
2016-05-17 22:36:35,079 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:35,079 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BE38BD61DE67449CB4516A3E9B90B5BB
2016-05-17 22:36:35,079 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB]
2016-05-17 22:36:35,080 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:35,080 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594884_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524594884_0001_1_00, startRequestedTime=1463524595079, startedTime=1463524595079
2016-05-17 22:36:35,080 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524594884_0001, with dag id: dag_1463524594884_0001_1
2016-05-17 22:36:35,080 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:35,081 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:35,082 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594884_0001_1][Event:TASK_STARTED]: vertexName=BE38BD61DE67449CB4516A3E9B90B5BB, taskId=task_1463524594884_0001_1_00_000000, scheduledTime=1463524595081, launchTime=1463524595081
2016-05-17 22:36:35,082 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524594884_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:35,082 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB]
2016-05-17 22:36:35,083 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [FA50ECBFE3414A188A739104E5AABC1C/ECFFCADB77994AA3A32078C9BD470F14] each/(1/1) ...yplatform/-1_ARGS_RESULTS, VertexName: BE38BD61DE67449CB4516A3E9B90B5BB, VertexParallelism: 1, TaskAttemptID:attempt_1463524594884_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0C63A9F508D04B0583572246464AF2A7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=1F2779593D19445EACAA3CB4E43715D7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:35,083 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524594884_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:35,083 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:35,084 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:35,085 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:35,085 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:35,086 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524594884_0001_00_000001
2016-05-17 22:36:35,086 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524594884_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:35,089 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524594884_0001_00_000001
2016-05-17 22:36:35,089 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:35,089 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:35,090 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594884_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524594884_0001_00_000001, launchTime=1463524595089
2016-05-17 22:36:35,090 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524594884_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:35,090 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524594884_0001_00_000001
2016-05-17 22:36:35,092 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524594884_0001_1_00_000000_0] to container: [container_1463524594884_0001_00_000001]
2016-05-17 22:36:35,092 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524594884_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:35,092 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524594884_0001_1_00_000000_0] started. Is using containerId: [container_1463524594884_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:35,093 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:35,093 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594884_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BE38BD61DE67449CB4516A3E9B90B5BB, taskAttemptId=attempt_1463524594884_0001_1_00_000000_0, startTime=1463524595092, containerId=container_1463524594884_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524594884_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:35,093 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524594884_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:35,093 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524594884_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:35,094 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524594884_0001_00_000001 given task: attempt_1463524594884_0001_1_00_000000_0
2016-05-17 22:36:35,094 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524594884_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524594884_0001_1_00_000000_0
2016-05-17 22:36:35 Starting to run new task attempt: attempt_1463524594884_0001_1_00_000000_0
2016-05-17 22:36:35,094 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524594884_0001_1_00_000000_0
2016-05-17 22:36:35,094 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:35,095 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:35,095 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:35,095 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:35,095 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:35,095 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:35,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FA50ECBFE3414A188A739104E5AABC1C/ECFFCADB77994AA3A32078C9BD470F14] each/(1/1) ...yplatform/-1_ARGS_RESULTS, VertexName: BE38BD61DE67449CB4516A3E9B90B5BB, VertexParallelism: 1, TaskAttemptID:attempt_1463524594884_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0C63A9F508D04B0583572246464AF2A7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=1F2779593D19445EACAA3CB4E43715D7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:35,095 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:35,096 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524594884_0001_1_00_000000_0
2016-05-17 22:36:35,096 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:35,096 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:35,098 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=0C63A9F508D04B0583572246464AF2A7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:35,099 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:35,099 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 0C63A9F508D04B0583572246464AF2A7
2016-05-17 22:36:35,100 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 0C63A9F508D04B0583572246464AF2A7
2016-05-17 22:36:35,103 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:35,104 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:35,105 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 0C63A9F508D04B0583572246464AF2A7
2016-05-17 22:36:35,105 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 0C63A9F508D04B0583572246464AF2A7
2016-05-17 22:36:35,105 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=1F2779593D19445EACAA3CB4E43715D7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:35,105 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524594884_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:35,107 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:35,107 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 1F2779593D19445EACAA3CB4E43715D7
2016-05-17 22:36:35,108 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:35,108 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:35,108 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 1F2779593D19445EACAA3CB4E43715D7
2016-05-17 22:36:35,117 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BE38BD61DE67449CB4516A3E9B90B5BB, ordinal: 0
2016-05-17 22:36:35,117 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BE38BD61DE67449CB4516A3E9B90B5BB, mem on start (mb), free: 183, total: 387, max: 672
2016-05-17 22:36:35,117 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:35,117 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:35,118 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:35,122 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:35,123 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 1F2779593D19445EACAA3CB4E43715D7
2016-05-17 22:36:35,123 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:35,123 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:35,124 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:35,124 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:35,124 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:35,125 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:35,125 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:35,125 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:35,126 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 0C63A9F508D04B0583572246464AF2A7, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:35,126 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 1F2779593D19445EACAA3CB4E43715D7, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:35,126 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:35,126 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 0C63A9F508D04B0583572246464AF2A7
2016-05-17 22:36:35,127 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:35,127 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 0C63A9F508D04B0583572246464AF2A7
2016-05-17 22:36:35,127 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 0C63A9F508D04B0583572246464AF2A7 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:35,128 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:35,128 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:35,128 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:35,129 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524594884_0001_1_00_000000_0
2016-05-17 22:36:35,132 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:35,132 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 0C63A9F508D04B0583572246464AF2A7
2016-05-17 22:36:35,133 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_ARGS_RESULTS"], id: 1F2779593D19445EACAA3CB4E43715D7
2016-05-17 22:36:35,133 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_ARGS_RESULTS"]
2016-05-17 22:36:35,133 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:35,133 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:35,133 INFO input.MRInput (MRInput.java:checkAndAwaitRecordReaderInitialization(597)) - Awaiting RecordReader initialization
2016-05-17 22:36:35,133 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:35,135 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@36a0e964
2016-05-17 22:36:35,137 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@36a0e964
2016-05-17 22:36:35,137 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:35,137 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:35,138 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BE38BD61DE67449CB4516A3E9B90B5BB, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:35,141 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:35,141 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:35,142 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524594884_0001_1_00_000000_0
2016-05-17 22:36:35,142 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524594884_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:35,143 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524594884_0001_1_00_000000_0
2016-05-17 22:36:35,143 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BE38BD61DE67449CB4516A3E9B90B5BB, mem on close (mb), free: 181, total: 387, max: 672
2016-05-17 22:36:35 Completed running task attempt: attempt_1463524594884_0001_1_00_000000_0
2016-05-17 22:36:35,144 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524594884_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:35,145 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406323200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524595138, Process_Duration=5, Process_End_Time=1463524595143, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:35,146 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594884_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BE38BD61DE67449CB4516A3E9B90B5BB, taskAttemptId=attempt_1463524594884_0001_1_00_000000_0, startTime=1463524595092, finishTime=1463524595145, timeTaken=53, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406323200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524595138, Process_Duration=5, Process_End_Time=1463524595143, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:35,146 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524594884_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:35,146 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524594884_0001_1_00_000000_0
2016-05-17 22:36:35,146 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:35,147 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:35,147 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594884_0001_1][Event:TASK_FINISHED]: vertexName=BE38BD61DE67449CB4516A3E9B90B5BB, taskId=task_1463524594884_0001_1_00_000000, startTime=1463524595092, finishTime=1463524595146, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1463524594884_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406323200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524595138, Process_Duration=5, Process_End_Time=1463524595143, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:35,147 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524594884_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:35,147 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB] : 1
2016-05-17 22:36:35,147 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB]
2016-05-17 22:36:35,149 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:35,149 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:35,149 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524594884_0001_00_000001
2016-05-17 22:36:35,150 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594884_0001_1][Event:VERTEX_FINISHED]: vertexName=BE38BD61DE67449CB4516A3E9B90B5BB, vertexId=vertex_1463524594884_0001_1_00, initRequestedTime=1463524595061, initedTime=1463524595077, startRequestedTime=1463524595079, startedTime=1463524595079, finishTime=1463524595148, timeTaken=69, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406323200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524595138, Process_Duration=5, Process_End_Time=1463524595143, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524595092, firstTasksToStart=[ task_1463524594884_0001_1_00_000000 ], lastTaskFinishTime=1463524595145, lastTasksToFinish=[ task_1463524594884_0001_1_00_000000 ], minTaskDuration=53, maxTaskDuration=53, avgTaskDuration=53.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524594884_0001_1_00_000000 ], longestDurationTasks=[ task_1463524594884_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:35,151 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:35,151 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:35,151 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524594884_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:35,152 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:35,152 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594884_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524594884_0001_00_000001, stoppedTime=1463524595152, exitStatus=0
2016-05-17 22:36:35,153 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524594884_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:35,153 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:35,153 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:35,154 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524594884_0001_1
2016-05-17 22:36:35,154 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594884_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524594884_0001_1
2016-05-17 22:36:35,155 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 1F2779593D19445EACAA3CB4E43715D7 for vertex: vertex_1463524594884_0001_1_00 [BE38BD61DE67449CB4516A3E9B90B5BB]
2016-05-17 22:36:35,160 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524594884_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:35,160 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524594884_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:35,161 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524594884_0001_00_000001, Exiting
2016-05-17 22:36:35,161 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524594884_0001_00_000001
2016-05-17 22:36:35,161 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524594884_0001_00_000001
2016-05-17 22:36:35,166 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524594884_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:35,166 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524594884_0001_00_000001 completed successfully
2016-05-17 22:36:35,168 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524594884_0001_1, queueSize=0
2016-05-17 22:36:35,169 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:35,170 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524594884_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524594884_0001_1, startTime=1463524595058, finishTime=1463524595167, timeTaken=109, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406323200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524595138, Process_Duration=5, Process_End_Time=1463524595143, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:35,170 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524594884_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:35,170 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524594884_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:35,171 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524594884_0001_00_000001
2016-05-17 22:36:35,171 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524594884_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:35 Completed Dag: dag_1463524594884_0001_1
2016-05-17 22:36:35,173 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524594884_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:35,174 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524594884_0001_1_post
2016-05-17 22:36:35,174 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:35,175 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:35,179 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/-1_ARGS_RESULTS, applicationId=application_1463524594884_0001
2016-05-17 22:36:35,181 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:35,181 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:35,181 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:35,181 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:35,181 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:35,182 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:35,183 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:35,183 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:35,183 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:35,183 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:35,183 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:35,184 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:35,184 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:35,188 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/FA50ECBFE3414A188A739104E5AABC1C/ECFFCADB77994AA3A32078C9BD470F14/.tez/application_1463524594884_0001
2016-05-17 22:36:35,188 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_ARGS_RESULTS/_temporary
2016-05-17 22:36:35,189 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.324
2016-05-17 22:36:35,258 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_ARGS_RESULTS
2016-05-17 22:36:35,280 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:35,281 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:35,312 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-17 22:36:35,316 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.008
2016-05-17 22:36:35,316 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:35,316 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:35,317 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_ARGS_ALL
2016-05-17 22:36:35,338 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:35,338 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:35,387 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-17 22:36:35,396 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-17 22:36:35,396 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:35,396 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:35,397 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:35,412 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:35,413 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_ARGS_REPLACE
2016-05-17 22:36:35,421 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:35,422 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:35,422 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_ARGS_REPLACE"]
2016-05-17 22:36:35,423 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:35,423 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:35,423 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:35,424 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...yplatform/-1_ARGS_REPLACE
2016-05-17 22:36:35,424 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:35,437 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:35,443 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:35,443 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:35,443 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:35,452 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/19C8D15B99E3408BBCCAA01F6EAB8377/BB966091A78B4934A5962237C0957D8F/.tez/application_1463524595443_0001 doesn't exist and is created
2016-05-17 22:36:35,467 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:35,468 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:35,468 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/19C8D15B99E3408BBCCAA01F6EAB8377/BB966091A78B4934A5962237C0957D8F/.tez/application_1463524595443_0001
2016-05-17 22:36:35,471 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524595443_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:35,482 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:35,483 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:35,484 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:35,485 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:35,485 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/19C8D15B99E3408BBCCAA01F6EAB8377/BB966091A78B4934A5962237C0957D8F/.tez/application_1463524595443_0001/localmode-log-dir
2016-05-17 22:36:35,485 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/19C8D15B99E3408BBCCAA01F6EAB8377/BB966091A78B4934A5962237C0957D8F/.tez/application_1463524595443_0001/localmode-log-dir/history.txt.appattempt_1463524595443_0001_000000, maxErrors=10
2016-05-17 22:36:35,485 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:35,485 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524595443_0001_000000, appSubmitTime=1463524595471, launchTime=1463524595471
2016-05-17 22:36:35,489 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:35,491 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:35,492 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:35,493 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:35,495 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:35424
2016-05-17 22:36:35,496 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:35,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524595443_0001_000000, startTime=1463524595498
2016-05-17 22:36:35,498 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:35,498 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:35,549 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:35,549 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:35,568 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:35,568 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:35,568 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/-1_ARGS_REPLACE, applicationId=application_1463524595443_0001, dagName=[19C8D15B99E3408BBCCAA01F6EAB8377/BB966091A78B4934A5962237C0957D8F] each/(1/1) ...yplatform/-1_ARGS_REPLACE
2016-05-17 22:36:35,585 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [19C8D15B99E3408BBCCAA01F6EAB8377/BB966091A78B4934A5962237C0957D8F] each/(1/1) ...yplatform/-1_ARGS_REPLACE
2016-05-17 22:36:35,586 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524595443_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/19C8D15B99E3408BBCCAA01F6EAB8377/BB966091A78B4934A5962237C0957D8F/.tez/application_1463524595443_0001/localmode-log-dir/dag_1463524595443_0001_1.dot
2016-05-17 22:36:35,587 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/19C8D15B99E3408BBCCAA01F6EAB8377/BB966091A78B4934A5962237C0957D8F/.tez/application_1463524595443_0001/localmode-log-dir/dag_1463524595443_0001_1-tez-dag.pb.txt
2016-05-17 22:36:35,591 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524595443_0001_1
2016-05-17 22:36:35,591 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:35,591 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [19C8D15B99E3408BBCCAA01F6EAB8377/BB966091A78B4934A5962237C0957D8F] each/(1/1) ...yplatform/-1_ARGS_REPLACE
2016-05-17 22:36:35 Running Dag: dag_1463524595443_0001_1
2016-05-17 22:36:35,602 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595443_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524595443_0001_1, submitTime=1463524595585
2016-05-17 22:36:35,602 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:35,603 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C53226632A7B4A728A74EB9BAE0E7BF5
2016-05-17 22:36:35,603 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:35,603 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595443_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524595443_0001_1, initTime=1463524595603
2016-05-17 22:36:35,603 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:35,605 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524595443_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:35,605 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595443_0001_1][Event:DAG_STARTED]: dagID=dag_1463524595443_0001_1, startTime=1463524595605
2016-05-17 22:36:35,605 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:35,605 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524595443_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:35,605 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/-1_ARGS_REPLACE, applicationId=application_1463524595443_0001, dagName=[19C8D15B99E3408BBCCAA01F6EAB8377/BB966091A78B4934A5962237C0957D8F] each/(1/1) ...yplatform/-1_ARGS_REPLACE
2016-05-17 22:36:35,606 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C53226632A7B4A728A74EB9BAE0E7BF5 : {14880A1ADABF44558ADCFFA5C28869D0={InputName=14880A1ADABF44558ADCFFA5C28869D0}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:35,606 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 14880A1ADABF44558ADCFFA5C28869D0, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:35,606 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5]
2016-05-17 22:36:35,606 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5]
2016-05-17 22:36:35,606 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5]
2016-05-17 22:36:35,606 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5]. Starting root input initializers: 1
2016-05-17 22:36:35,607 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:35,607 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:35,607 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 14880A1ADABF44558ADCFFA5C28869D0 on vertex vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5]
2016-05-17 22:36:35,609 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:35,609 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 14880A1ADABF44558ADCFFA5C28869D0 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:35,609 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:35,610 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:35,611 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:35,611 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 14880A1ADABF44558ADCFFA5C28869D0 on vertex vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5]
2016-05-17 22:36:35,612 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:35,612 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {14880A1ADABF44558ADCFFA5C28869D0=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:35,612 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5] parallelism set to 1
2016-05-17 22:36:35,612 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:35,612 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595443_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524595443_0001_1_00, eventCount=1
2016-05-17 22:36:35,612 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5]
2016-05-17 22:36:35,613 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=38807D8D9CD84324956FFB68BC0FB599, vertexId=vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:35,613 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=38807D8D9CD84324956FFB68BC0FB599, vertexId=vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5]
2016-05-17 22:36:35,615 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=38807D8D9CD84324956FFB68BC0FB599, vertexName=C53226632A7B4A728A74EB9BAE0E7BF5, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:35,615 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=38807D8D9CD84324956FFB68BC0FB599, vertexId=vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5]
2016-05-17 22:36:35,616 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595443_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C53226632A7B4A728A74EB9BAE0E7BF5, vertexId=vertex_1463524595443_0001_1_00, initRequestedTime=1463524595606, initedTime=1463524595616, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:35,616 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5] with distanceFromRoot: 0
2016-05-17 22:36:35,616 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:35,616 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:35,617 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C53226632A7B4A728A74EB9BAE0E7BF5
2016-05-17 22:36:35,617 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5]
2016-05-17 22:36:35,617 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595443_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524595443_0001_1_00, startRequestedTime=1463524595617, startedTime=1463524595617
2016-05-17 22:36:35,617 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:35,617 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:35,618 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:35,619 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595443_0001_1][Event:TASK_STARTED]: vertexName=C53226632A7B4A728A74EB9BAE0E7BF5, taskId=task_1463524595443_0001_1_00_000000, scheduledTime=1463524595617, launchTime=1463524595617
2016-05-17 22:36:35,619 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524595443_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:35,619 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5]
2016-05-17 22:36:35,619 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [19C8D15B99E3408BBCCAA01F6EAB8377/BB966091A78B4934A5962237C0957D8F] each/(1/1) ...yplatform/-1_ARGS_REPLACE, VertexName: C53226632A7B4A728A74EB9BAE0E7BF5, VertexParallelism: 1, TaskAttemptID:attempt_1463524595443_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=14880A1ADABF44558ADCFFA5C28869D0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=38807D8D9CD84324956FFB68BC0FB599, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:35,619 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524595443_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:35,619 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:35,621 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:35,621 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:35,622 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:35,622 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524595443_0001_00_000001
2016-05-17 22:36:35,622 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524595443_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:35,624 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524595443_0001_00_000001
2016-05-17 22:36:35,624 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:35,625 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524595443_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:35,625 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524595443_0001, with dag id: dag_1463524595443_0001_1
2016-05-17 22:36:35,625 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:35,626 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595443_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524595443_0001_00_000001, launchTime=1463524595625
2016-05-17 22:36:35,627 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524595443_0001_00_000001
2016-05-17 22:36:35,628 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524595443_0001_1_00_000000_0] to container: [container_1463524595443_0001_00_000001]
2016-05-17 22:36:35,628 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524595443_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:35,628 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524595443_0001_1_00_000000_0] started. Is using containerId: [container_1463524595443_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:35,629 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:35,629 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595443_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C53226632A7B4A728A74EB9BAE0E7BF5, taskAttemptId=attempt_1463524595443_0001_1_00_000000_0, startTime=1463524595628, containerId=container_1463524595443_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524595443_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:35,629 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524595443_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:35,629 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524595443_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:35,631 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524595443_0001_00_000001 given task: attempt_1463524595443_0001_1_00_000000_0
2016-05-17 22:36:35,631 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524595443_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524595443_0001_1_00_000000_0
2016-05-17 22:36:35 Starting to run new task attempt: attempt_1463524595443_0001_1_00_000000_0
2016-05-17 22:36:35,632 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524595443_0001_1_00_000000_0
2016-05-17 22:36:35,632 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:35,632 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:35,632 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:35,632 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:35,632 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:35,632 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:35,632 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [19C8D15B99E3408BBCCAA01F6EAB8377/BB966091A78B4934A5962237C0957D8F] each/(1/1) ...yplatform/-1_ARGS_REPLACE, VertexName: C53226632A7B4A728A74EB9BAE0E7BF5, VertexParallelism: 1, TaskAttemptID:attempt_1463524595443_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=14880A1ADABF44558ADCFFA5C28869D0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=38807D8D9CD84324956FFB68BC0FB599, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:35,632 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:35,633 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524595443_0001_1_00_000000_0
2016-05-17 22:36:35,634 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524595443_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:35,634 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:35,634 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:35,635 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=14880A1ADABF44558ADCFFA5C28869D0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:35,635 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:35,635 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=38807D8D9CD84324956FFB68BC0FB599, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:35,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:35,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 38807D8D9CD84324956FFB68BC0FB599
2016-05-17 22:36:35,636 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:35,637 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 38807D8D9CD84324956FFB68BC0FB599
2016-05-17 22:36:35,638 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:35,644 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:35,644 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 14880A1ADABF44558ADCFFA5C28869D0
2016-05-17 22:36:35,644 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 14880A1ADABF44558ADCFFA5C28869D0
2016-05-17 22:36:35,649 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:35,650 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 14880A1ADABF44558ADCFFA5C28869D0
2016-05-17 22:36:35,650 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:35,650 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 38807D8D9CD84324956FFB68BC0FB599
2016-05-17 22:36:35,650 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 14880A1ADABF44558ADCFFA5C28869D0
2016-05-17 22:36:35,654 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C53226632A7B4A728A74EB9BAE0E7BF5, ordinal: 0
2016-05-17 22:36:35,655 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C53226632A7B4A728A74EB9BAE0E7BF5, mem on start (mb), free: 303, total: 388, max: 672
2016-05-17 22:36:35,655 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:35,655 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:35,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:35,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:35,656 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:35,657 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:35,657 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:35,658 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:35,658 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:35,659 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:35,659 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:35,659 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 38807D8D9CD84324956FFB68BC0FB599, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:35,660 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 14880A1ADABF44558ADCFFA5C28869D0, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:35,660 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:35,661 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 14880A1ADABF44558ADCFFA5C28869D0
2016-05-17 22:36:35,661 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:35,661 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 14880A1ADABF44558ADCFFA5C28869D0
2016-05-17 22:36:35,661 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 14880A1ADABF44558ADCFFA5C28869D0 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:35,662 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:35,662 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:35,662 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:35,663 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524595443_0001_1_00_000000_0
2016-05-17 22:36:35,663 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:35,663 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@abe36ee
2016-05-17 22:36:35,664 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@abe36ee
2016-05-17 22:36:35,664 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:35,664 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:35,667 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:35,667 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 14880A1ADABF44558ADCFFA5C28869D0
2016-05-17 22:36:35,668 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_ARGS_REPLACE"], id: 38807D8D9CD84324956FFB68BC0FB599
2016-05-17 22:36:35,668 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_ARGS_REPLACE"]
2016-05-17 22:36:35,668 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:35,669 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:35,669 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C53226632A7B4A728A74EB9BAE0E7BF5, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:35,672 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:35,672 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:35,673 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524595443_0001_1_00_000000_0
2016-05-17 22:36:35,673 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524595443_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:35,674 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524595443_0001_1_00_000000_0
2016-05-17 22:36:35,674 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C53226632A7B4A728A74EB9BAE0E7BF5, mem on close (mb), free: 302, total: 388, max: 672
2016-05-17 22:36:35 Completed running task attempt: attempt_1463524595443_0001_1_00_000000_0
2016-05-17 22:36:35,675 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524595443_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:35,676 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:35,676 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595443_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C53226632A7B4A728A74EB9BAE0E7BF5, taskAttemptId=attempt_1463524595443_0001_1_00_000000_0, startTime=1463524595628, finishTime=1463524595676, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524595669, Process_Duration=5, Process_End_Time=1463524595674, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:35,676 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524595443_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:35,676 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524595443_0001_1_00_000000_0
2016-05-17 22:36:35,677 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:35,677 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:35,677 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595443_0001_1][Event:TASK_FINISHED]: vertexName=C53226632A7B4A728A74EB9BAE0E7BF5, taskId=task_1463524595443_0001_1_00_000000, startTime=1463524595628, finishTime=1463524595677, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1463524595443_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524595669, Process_Duration=5, Process_End_Time=1463524595674, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:35,678 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524595443_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:35,678 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5] : 1
2016-05-17 22:36:35,678 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5]
2016-05-17 22:36:35,679 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:35,679 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595443_0001_1][Event:VERTEX_FINISHED]: vertexName=C53226632A7B4A728A74EB9BAE0E7BF5, vertexId=vertex_1463524595443_0001_1_00, initRequestedTime=1463524595606, initedTime=1463524595616, startRequestedTime=1463524595617, startedTime=1463524595617, finishTime=1463524595678, timeTaken=61, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524595669, Process_Duration=5, Process_End_Time=1463524595674, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524595628, firstTasksToStart=[ task_1463524595443_0001_1_00_000000 ], lastTaskFinishTime=1463524595676, lastTasksToFinish=[ task_1463524595443_0001_1_00_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524595443_0001_1_00_000000 ], longestDurationTasks=[ task_1463524595443_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:35,679 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:35,680 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524595669, Process_Duration=5, Process_End_Time=1463524595674, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:35,680 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524595443_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:35,681 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:35,681 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:35,681 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:35,681 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595443_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524595443_0001_00_000001, stoppedTime=1463524595681, exitStatus=0
2016-05-17 22:36:35,681 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524595443_0001_00_000001
2016-05-17 22:36:35,682 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524595443_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:35,682 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:35,682 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524595443_0001_1
2016-05-17 22:36:35,683 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595443_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524595443_0001_1
2016-05-17 22:36:35,683 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 38807D8D9CD84324956FFB68BC0FB599 for vertex: vertex_1463524595443_0001_1_00 [C53226632A7B4A728A74EB9BAE0E7BF5]
2016-05-17 22:36:35,683 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524595443_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:35,683 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524595443_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:35,684 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524595443_0001_00_000001, Exiting
2016-05-17 22:36:35,684 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524595443_0001_00_000001
2016-05-17 22:36:35,684 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524595443_0001_00_000001
2016-05-17 22:36:35,684 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524595443_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:35,685 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524595443_0001_00_000001 completed successfully
2016-05-17 22:36:35,692 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524595443_0001_1, queueSize=0
2016-05-17 22:36:35,692 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:35,693 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595443_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524595443_0001_1, startTime=1463524595605, finishTime=1463524595691, timeTaken=86, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524595669, Process_Duration=5, Process_End_Time=1463524595674, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:35,693 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524595443_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:35,693 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524595443_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:35,693 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524595443_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:35,693 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524595443_0001_00_000001
2016-05-17 22:36:35 Completed Dag: dag_1463524595443_0001_1
2016-05-17 22:36:35,695 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524595443_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:35,695 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524595443_0001_1_post
2016-05-17 22:36:35,695 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:35,695 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:35,700 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/-1_ARGS_REPLACE, applicationId=application_1463524595443_0001
2016-05-17 22:36:35,701 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:35,701 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:35,702 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:35,702 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:35,702 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:35,702 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:35,702 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:35,703 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:35,703 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:35,703 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:35,704 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:35,704 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:35,704 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:35,707 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/19C8D15B99E3408BBCCAA01F6EAB8377/BB966091A78B4934A5962237C0957D8F/.tez/application_1463524595443_0001
2016-05-17 22:36:35,708 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_ARGS_REPLACE/_temporary
2016-05-17 22:36:35,708 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.287
2016-05-17 22:36:35,772 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_ARGS_REPLACE
2016-05-17 22:36:35,796 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:35,796 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:35,829 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 22:36:35,836 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-17 22:36:35,837 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:35,837 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:35,837 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:35,851 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:35,851 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_ARGS_SWAP
2016-05-17 22:36:35,860 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:35,860 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:35,860 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_ARGS_SWAP"]
2016-05-17 22:36:35,860 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:35,861 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:35,862 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:35,862 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...mblyplatform/-1_ARGS_SWAP
2016-05-17 22:36:35,862 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:35,875 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:35,881 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:35,881 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:35,881 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:35,891 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/59D9E5EA9505483D840B954C65251767/C8857CEFF35748A0A52BB7E33CA05029/.tez/application_1463524595881_0001 doesn't exist and is created
2016-05-17 22:36:35,908 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:35,908 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:35,909 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/59D9E5EA9505483D840B954C65251767/C8857CEFF35748A0A52BB7E33CA05029/.tez/application_1463524595881_0001
2016-05-17 22:36:35,912 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524595881_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:35,923 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:35,924 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:35,926 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:35,926 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:35,927 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/59D9E5EA9505483D840B954C65251767/C8857CEFF35748A0A52BB7E33CA05029/.tez/application_1463524595881_0001/localmode-log-dir
2016-05-17 22:36:35,927 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/59D9E5EA9505483D840B954C65251767/C8857CEFF35748A0A52BB7E33CA05029/.tez/application_1463524595881_0001/localmode-log-dir/history.txt.appattempt_1463524595881_0001_000000, maxErrors=10
2016-05-17 22:36:35,927 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:35,928 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524595881_0001_000000, appSubmitTime=1463524595911, launchTime=1463524595911
2016-05-17 22:36:35,933 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:35,936 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:35,942 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:35,942 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:35,945 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:43475
2016-05-17 22:36:35,947 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:35,950 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524595881_0001_000000, startTime=1463524595950
2016-05-17 22:36:35,951 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:35,951 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:36,008 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:36,009 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:36,009 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...mblyplatform/-1_ARGS_SWAP, applicationId=application_1463524595881_0001, dagName=[59D9E5EA9505483D840B954C65251767/C8857CEFF35748A0A52BB7E33CA05029] each/(1/1) ...mblyplatform/-1_ARGS_SWAP
2016-05-17 22:36:36,026 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [59D9E5EA9505483D840B954C65251767/C8857CEFF35748A0A52BB7E33CA05029] each/(1/1) ...mblyplatform/-1_ARGS_SWAP
2016-05-17 22:36:36,027 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524595881_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/59D9E5EA9505483D840B954C65251767/C8857CEFF35748A0A52BB7E33CA05029/.tez/application_1463524595881_0001/localmode-log-dir/dag_1463524595881_0001_1.dot
2016-05-17 22:36:36,027 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/59D9E5EA9505483D840B954C65251767/C8857CEFF35748A0A52BB7E33CA05029/.tez/application_1463524595881_0001/localmode-log-dir/dag_1463524595881_0001_1-tez-dag.pb.txt
2016-05-17 22:36:36,032 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524595881_0001_1
2016-05-17 22:36:36,032 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:36,032 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [59D9E5EA9505483D840B954C65251767/C8857CEFF35748A0A52BB7E33CA05029] each/(1/1) ...mblyplatform/-1_ARGS_SWAP
2016-05-17 22:36:36 Running Dag: dag_1463524595881_0001_1
2016-05-17 22:36:36,044 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595881_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524595881_0001_1, submitTime=1463524596026
2016-05-17 22:36:36,044 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:36,044 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A4C908F1C4684D52BFE45E8CB300872D
2016-05-17 22:36:36,044 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:36,045 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:36,045 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:36,045 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595881_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524595881_0001_1, initTime=1463524596044
2016-05-17 22:36:36,045 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:36,045 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524595881_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:36,046 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595881_0001_1][Event:DAG_STARTED]: dagID=dag_1463524595881_0001_1, startTime=1463524596046
2016-05-17 22:36:36,046 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:36,046 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524595881_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:36,046 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A4C908F1C4684D52BFE45E8CB300872D : {4C373B0E0CAD4964B31B8CB781B60EA3={InputName=4C373B0E0CAD4964B31B8CB781B60EA3}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:36,046 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 4C373B0E0CAD4964B31B8CB781B60EA3, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:36,046 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D]
2016-05-17 22:36:36,047 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D]
2016-05-17 22:36:36,047 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D]
2016-05-17 22:36:36,047 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D]. Starting root input initializers: 1
2016-05-17 22:36:36,048 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:36,048 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 4C373B0E0CAD4964B31B8CB781B60EA3 on vertex vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D]
2016-05-17 22:36:36,048 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:36,048 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...mblyplatform/-1_ARGS_SWAP, applicationId=application_1463524595881_0001, dagName=[59D9E5EA9505483D840B954C65251767/C8857CEFF35748A0A52BB7E33CA05029] each/(1/1) ...mblyplatform/-1_ARGS_SWAP
2016-05-17 22:36:36,050 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:36,050 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 4C373B0E0CAD4964B31B8CB781B60EA3 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:36,050 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:36,050 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:36,051 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:36,052 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 4C373B0E0CAD4964B31B8CB781B60EA3 on vertex vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D]
2016-05-17 22:36:36,052 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:36,052 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {4C373B0E0CAD4964B31B8CB781B60EA3=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:36,052 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D] parallelism set to 1
2016-05-17 22:36:36,052 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:36,052 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595881_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524595881_0001_1_00, eventCount=1
2016-05-17 22:36:36,053 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D]
2016-05-17 22:36:36,053 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=7F7AD9C0FD3C4267BC9AB0B21437C4EC, vertexId=vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:36,053 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=7F7AD9C0FD3C4267BC9AB0B21437C4EC, vertexId=vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D]
2016-05-17 22:36:36,054 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=7F7AD9C0FD3C4267BC9AB0B21437C4EC, vertexName=A4C908F1C4684D52BFE45E8CB300872D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:36,055 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=7F7AD9C0FD3C4267BC9AB0B21437C4EC, vertexId=vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D]
2016-05-17 22:36:36,056 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595881_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A4C908F1C4684D52BFE45E8CB300872D, vertexId=vertex_1463524595881_0001_1_00, initRequestedTime=1463524596046, initedTime=1463524596055, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:36,056 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:36,056 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D] with distanceFromRoot: 0
2016-05-17 22:36:36,056 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:36,056 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A4C908F1C4684D52BFE45E8CB300872D
2016-05-17 22:36:36,057 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D]
2016-05-17 22:36:36,057 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:36,057 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595881_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524595881_0001_1_00, startRequestedTime=1463524596056, startedTime=1463524596056
2016-05-17 22:36:36,057 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:36,058 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:36,058 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595881_0001_1][Event:TASK_STARTED]: vertexName=A4C908F1C4684D52BFE45E8CB300872D, taskId=task_1463524595881_0001_1_00_000000, scheduledTime=1463524596058, launchTime=1463524596058
2016-05-17 22:36:36,058 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524595881_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:36,059 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D]
2016-05-17 22:36:36,059 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [59D9E5EA9505483D840B954C65251767/C8857CEFF35748A0A52BB7E33CA05029] each/(1/1) ...mblyplatform/-1_ARGS_SWAP, VertexName: A4C908F1C4684D52BFE45E8CB300872D, VertexParallelism: 1, TaskAttemptID:attempt_1463524595881_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4C373B0E0CAD4964B31B8CB781B60EA3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=7F7AD9C0FD3C4267BC9AB0B21437C4EC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:36,059 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524595881_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:36,059 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:36,060 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:36,060 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:36,061 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:36,061 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524595881_0001_00_000001
2016-05-17 22:36:36,061 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524595881_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:36,063 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524595881_0001_00_000001
2016-05-17 22:36:36,063 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:36,063 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:36,064 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524595881_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:36,064 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595881_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524595881_0001_00_000001, launchTime=1463524596063
2016-05-17 22:36:36,064 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524595881_0001_00_000001
2016-05-17 22:36:36,065 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524595881_0001_1_00_000000_0] to container: [container_1463524595881_0001_00_000001]
2016-05-17 22:36:36,065 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524595881_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:36,065 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524595881_0001_00_000001 given task: attempt_1463524595881_0001_1_00_000000_0
2016-05-17 22:36:36,065 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524595881_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524595881_0001_1_00_000000_0
2016-05-17 22:36:36,066 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524595881_0001_1_00_000000_0] started. Is using containerId: [container_1463524595881_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:36,066 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:36,066 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595881_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A4C908F1C4684D52BFE45E8CB300872D, taskAttemptId=attempt_1463524595881_0001_1_00_000000_0, startTime=1463524596065, containerId=container_1463524595881_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524595881_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:36,066 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524595881_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:36,066 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524595881_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:36 Starting to run new task attempt: attempt_1463524595881_0001_1_00_000000_0
2016-05-17 22:36:36,067 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524595881_0001_1_00_000000_0
2016-05-17 22:36:36,067 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:36,067 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:36,067 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:36,067 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:36,067 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:36,067 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:36,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [59D9E5EA9505483D840B954C65251767/C8857CEFF35748A0A52BB7E33CA05029] each/(1/1) ...mblyplatform/-1_ARGS_SWAP, VertexName: A4C908F1C4684D52BFE45E8CB300872D, VertexParallelism: 1, TaskAttemptID:attempt_1463524595881_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4C373B0E0CAD4964B31B8CB781B60EA3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=7F7AD9C0FD3C4267BC9AB0B21437C4EC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:36,068 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:36,074 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524595881_0001_1_00_000000_0
2016-05-17 22:36:36,074 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:36,074 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:36,075 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524595881_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:36,075 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=4C373B0E0CAD4964B31B8CB781B60EA3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:36,076 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:36,076 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:36,076 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 4C373B0E0CAD4964B31B8CB781B60EA3
2016-05-17 22:36:36,076 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 4C373B0E0CAD4964B31B8CB781B60EA3
2016-05-17 22:36:36,076 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=7F7AD9C0FD3C4267BC9AB0B21437C4EC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:36,077 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:36,077 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 7F7AD9C0FD3C4267BC9AB0B21437C4EC
2016-05-17 22:36:36,077 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:36,078 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 7F7AD9C0FD3C4267BC9AB0B21437C4EC
2016-05-17 22:36:36,081 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524595881_0001, with dag id: dag_1463524595881_0001_1
2016-05-17 22:36:36,081 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:36,086 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:36,086 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 4C373B0E0CAD4964B31B8CB781B60EA3
2016-05-17 22:36:36,086 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 4C373B0E0CAD4964B31B8CB781B60EA3
2016-05-17 22:36:36,093 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A4C908F1C4684D52BFE45E8CB300872D, ordinal: 0
2016-05-17 22:36:36,093 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A4C908F1C4684D52BFE45E8CB300872D, mem on start (mb), free: 227, total: 388, max: 672
2016-05-17 22:36:36,093 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:36,093 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:36,093 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:36,096 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:36,096 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 7F7AD9C0FD3C4267BC9AB0B21437C4EC
2016-05-17 22:36:36,097 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:36,097 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:36,097 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:36,097 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:36,097 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:36,097 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:36,099 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:36,099 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:36,099 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 4C373B0E0CAD4964B31B8CB781B60EA3, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:36,099 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 7F7AD9C0FD3C4267BC9AB0B21437C4EC, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:36,099 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:36,099 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 4C373B0E0CAD4964B31B8CB781B60EA3 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:36,099 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 4C373B0E0CAD4964B31B8CB781B60EA3
2016-05-17 22:36:36,099 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:36,099 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:36,099 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:36,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 4C373B0E0CAD4964B31B8CB781B60EA3
2016-05-17 22:36:36,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:36,100 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524595881_0001_1_00_000000_0
2016-05-17 22:36:36,101 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:36,102 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@44e49180
2016-05-17 22:36:36,103 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:36,103 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 4C373B0E0CAD4964B31B8CB781B60EA3
2016-05-17 22:36:36,103 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@44e49180
2016-05-17 22:36:36,103 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_ARGS_SWAP"], id: 7F7AD9C0FD3C4267BC9AB0B21437C4EC
2016-05-17 22:36:36,104 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_ARGS_SWAP"]
2016-05-17 22:36:36,105 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:36,105 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:36,105 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:36,105 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:36,105 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A4C908F1C4684D52BFE45E8CB300872D, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:36,109 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:36,109 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:36,110 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524595881_0001_1_00_000000_0
2016-05-17 22:36:36,110 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524595881_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:36,110 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524595881_0001_1_00_000000_0
2016-05-17 22:36:36,110 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A4C908F1C4684D52BFE45E8CB300872D, mem on close (mb), free: 225, total: 388, max: 672
2016-05-17 22:36:36 Completed running task attempt: attempt_1463524595881_0001_1_00_000000_0
2016-05-17 22:36:36,111 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524595881_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:36,112 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595881_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A4C908F1C4684D52BFE45E8CB300872D, taskAttemptId=attempt_1463524595881_0001_1_00_000000_0, startTime=1463524596065, finishTime=1463524596112, timeTaken=47, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524596106, Process_Duration=4, Process_End_Time=1463524596110, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:36,112 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524595881_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:36,112 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:36,112 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524595881_0001_1_00_000000_0
2016-05-17 22:36:36,112 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:36,114 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595881_0001_1][Event:TASK_FINISHED]: vertexName=A4C908F1C4684D52BFE45E8CB300872D, taskId=task_1463524595881_0001_1_00_000000, startTime=1463524596065, finishTime=1463524596113, timeTaken=48, status=SUCCEEDED, successfulAttemptID=attempt_1463524595881_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524596106, Process_Duration=4, Process_End_Time=1463524596110, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:36,114 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524595881_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:36,114 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D] : 1
2016-05-17 22:36:36,114 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D]
2016-05-17 22:36:36,114 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:36,115 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595881_0001_1][Event:VERTEX_FINISHED]: vertexName=A4C908F1C4684D52BFE45E8CB300872D, vertexId=vertex_1463524595881_0001_1_00, initRequestedTime=1463524596046, initedTime=1463524596055, startRequestedTime=1463524596056, startedTime=1463524596056, finishTime=1463524596114, timeTaken=58, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524596106, Process_Duration=4, Process_End_Time=1463524596110, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524596065, firstTasksToStart=[ task_1463524595881_0001_1_00_000000 ], lastTaskFinishTime=1463524596112, lastTasksToFinish=[ task_1463524595881_0001_1_00_000000 ], minTaskDuration=47, maxTaskDuration=47, avgTaskDuration=47.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524595881_0001_1_00_000000 ], longestDurationTasks=[ task_1463524595881_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:36,115 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:36,115 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:36,116 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524595881_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:36,116 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595881_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524595881_0001_00_000001, stoppedTime=1463524596116, exitStatus=0
2016-05-17 22:36:36,116 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524595881_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:36,116 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:36,116 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524595881_0001_1
2016-05-17 22:36:36,116 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595881_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524595881_0001_1
2016-05-17 22:36:36,116 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 7F7AD9C0FD3C4267BC9AB0B21437C4EC for vertex: vertex_1463524595881_0001_1_00 [A4C908F1C4684D52BFE45E8CB300872D]
2016-05-17 22:36:36,116 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:36,117 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524596106, Process_Duration=4, Process_End_Time=1463524596110, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:36,117 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:36,117 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:36,117 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524595881_0001_00_000001
2016-05-17 22:36:36,118 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524595881_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:36,118 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524595881_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:36,118 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524595881_0001_00_000001, Exiting
2016-05-17 22:36:36,118 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524595881_0001_00_000001
2016-05-17 22:36:36,118 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524595881_0001_00_000001
2016-05-17 22:36:36,120 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524595881_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:36,120 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524595881_0001_00_000001 completed successfully
2016-05-17 22:36:36,125 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524595881_0001_1, queueSize=0
2016-05-17 22:36:36,125 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:36,126 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524595881_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524595881_0001_1, startTime=1463524596046, finishTime=1463524596124, timeTaken=78, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524596106, Process_Duration=4, Process_End_Time=1463524596110, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:36,126 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524595881_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:36,126 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524595881_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:36,127 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524595881_0001_00_000001
2016-05-17 22:36:36,127 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524595881_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:36 Completed Dag: dag_1463524595881_0001_1
2016-05-17 22:36:36,128 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524595881_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:36,129 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524595881_0001_1_post
2016-05-17 22:36:36,129 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:36,129 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:36,134 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...mblyplatform/-1_ARGS_SWAP, applicationId=application_1463524595881_0001
2016-05-17 22:36:36,136 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:36,136 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:36,136 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:36,137 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:36,137 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:36,138 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:36,138 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:36,138 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:36,138 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:36,139 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:36,139 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:36,139 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:36,140 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:36,148 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/59D9E5EA9505483D840B954C65251767/C8857CEFF35748A0A52BB7E33CA05029/.tez/application_1463524595881_0001
2016-05-17 22:36:36,148 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/-1_ARGS_SWAP/_temporary
2016-05-17 22:36:36,149 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.288
2016-05-17 22:36:36,214 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//-1_ARGS_SWAP
2016-05-17 22:36:36,236 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:36,236 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:36,266 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 22:36:36,271 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 22:36:36,272 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:36,272 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:36,272 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:36,286 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:36,286 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_-1
2016-05-17 22:36:36,295 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:36,295 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:36,295 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_-1"]
2016-05-17 22:36:36,296 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:36,296 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:36,296 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:36,297 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...mblyplatform/0_'field'_-1
2016-05-17 22:36:36,297 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:36,309 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:36,313 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:36,314 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:36,314 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:36,323 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CB3947E7F8AF4D84B754A061919FF90D/A0C0ABEF8E464D639B95A18FE56D7A7E/.tez/application_1463524596314_0001 doesn't exist and is created
2016-05-17 22:36:36,339 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:36,339 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:36,339 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CB3947E7F8AF4D84B754A061919FF90D/A0C0ABEF8E464D639B95A18FE56D7A7E/.tez/application_1463524596314_0001
2016-05-17 22:36:36,342 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524596314_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:36,352 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:36,353 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:36,354 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:36,354 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:36,354 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CB3947E7F8AF4D84B754A061919FF90D/A0C0ABEF8E464D639B95A18FE56D7A7E/.tez/application_1463524596314_0001/localmode-log-dir
2016-05-17 22:36:36,354 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CB3947E7F8AF4D84B754A061919FF90D/A0C0ABEF8E464D639B95A18FE56D7A7E/.tez/application_1463524596314_0001/localmode-log-dir/history.txt.appattempt_1463524596314_0001_000000, maxErrors=10
2016-05-17 22:36:36,354 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:36,355 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524596314_0001_000000, appSubmitTime=1463524596342, launchTime=1463524596342
2016-05-17 22:36:36,358 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:36,359 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:36,360 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:36,361 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:56340
2016-05-17 22:36:36,362 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:36,364 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:36,366 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524596314_0001_000000, startTime=1463524596366
2016-05-17 22:36:36,366 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:36,367 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:36,439 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:36,439 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:36,439 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...mblyplatform/0_'field'_-1, applicationId=application_1463524596314_0001, dagName=[CB3947E7F8AF4D84B754A061919FF90D/A0C0ABEF8E464D639B95A18FE56D7A7E] each/(1/1) ...mblyplatform/0_'field'_-1
2016-05-17 22:36:36,461 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [CB3947E7F8AF4D84B754A061919FF90D/A0C0ABEF8E464D639B95A18FE56D7A7E] each/(1/1) ...mblyplatform/0_'field'_-1
2016-05-17 22:36:36,463 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524596314_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CB3947E7F8AF4D84B754A061919FF90D/A0C0ABEF8E464D639B95A18FE56D7A7E/.tez/application_1463524596314_0001/localmode-log-dir/dag_1463524596314_0001_1.dot
2016-05-17 22:36:36,463 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CB3947E7F8AF4D84B754A061919FF90D/A0C0ABEF8E464D639B95A18FE56D7A7E/.tez/application_1463524596314_0001/localmode-log-dir/dag_1463524596314_0001_1-tez-dag.pb.txt
2016-05-17 22:36:36,468 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524596314_0001_1
2016-05-17 22:36:36,468 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:36,469 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [CB3947E7F8AF4D84B754A061919FF90D/A0C0ABEF8E464D639B95A18FE56D7A7E] each/(1/1) ...mblyplatform/0_'field'_-1
2016-05-17 22:36:36 Running Dag: dag_1463524596314_0001_1
2016-05-17 22:36:36,482 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596314_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524596314_0001_1, submitTime=1463524596461
2016-05-17 22:36:36,482 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:36,482 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E2B2EA6480344D69A84410A164146CC8
2016-05-17 22:36:36,483 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:36,483 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596314_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524596314_0001_1, initTime=1463524596482
2016-05-17 22:36:36,484 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:36,484 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524596314_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:36,485 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596314_0001_1][Event:DAG_STARTED]: dagID=dag_1463524596314_0001_1, startTime=1463524596485
2016-05-17 22:36:36,485 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:36,485 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524596314_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:36,486 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...mblyplatform/0_'field'_-1, applicationId=application_1463524596314_0001, dagName=[CB3947E7F8AF4D84B754A061919FF90D/A0C0ABEF8E464D639B95A18FE56D7A7E] each/(1/1) ...mblyplatform/0_'field'_-1
2016-05-17 22:36:36,486 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E2B2EA6480344D69A84410A164146CC8 : {A699F028F1904A6ABC50C12C36A58900={InputName=A699F028F1904A6ABC50C12C36A58900}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:36,486 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A699F028F1904A6ABC50C12C36A58900, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:36,486 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8]
2016-05-17 22:36:36,486 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8]
2016-05-17 22:36:36,486 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8]
2016-05-17 22:36:36,486 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8]. Starting root input initializers: 1
2016-05-17 22:36:36,487 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:36,487 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:36,488 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A699F028F1904A6ABC50C12C36A58900 on vertex vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8]
2016-05-17 22:36:36,489 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:36,489 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A699F028F1904A6ABC50C12C36A58900 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:36,490 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:36,490 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:36,491 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:36,491 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A699F028F1904A6ABC50C12C36A58900 on vertex vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8]
2016-05-17 22:36:36,492 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:36,492 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A699F028F1904A6ABC50C12C36A58900=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:36,492 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8] parallelism set to 1
2016-05-17 22:36:36,492 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:36,493 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596314_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524596314_0001_1_00, eventCount=1
2016-05-17 22:36:36,493 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8]
2016-05-17 22:36:36,493 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A6651F91CC08479EAF7C236D62BEDA32, vertexId=vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:36,493 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A6651F91CC08479EAF7C236D62BEDA32, vertexId=vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8]
2016-05-17 22:36:36,495 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A6651F91CC08479EAF7C236D62BEDA32, vertexName=E2B2EA6480344D69A84410A164146CC8, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:36,495 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A6651F91CC08479EAF7C236D62BEDA32, vertexId=vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8]
2016-05-17 22:36:36,496 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:36,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596314_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E2B2EA6480344D69A84410A164146CC8, vertexId=vertex_1463524596314_0001_1_00, initRequestedTime=1463524596486, initedTime=1463524596496, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:36,497 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8] with distanceFromRoot: 0
2016-05-17 22:36:36,497 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:36,497 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E2B2EA6480344D69A84410A164146CC8
2016-05-17 22:36:36,498 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8]
2016-05-17 22:36:36,498 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:36,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596314_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524596314_0001_1_00, startRequestedTime=1463524596497, startedTime=1463524596497
2016-05-17 22:36:36,499 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:36,499 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:36,500 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596314_0001_1][Event:TASK_STARTED]: vertexName=E2B2EA6480344D69A84410A164146CC8, taskId=task_1463524596314_0001_1_00_000000, scheduledTime=1463524596499, launchTime=1463524596499
2016-05-17 22:36:36,500 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524596314_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:36,500 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8]
2016-05-17 22:36:36,500 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [CB3947E7F8AF4D84B754A061919FF90D/A0C0ABEF8E464D639B95A18FE56D7A7E] each/(1/1) ...mblyplatform/0_'field'_-1, VertexName: E2B2EA6480344D69A84410A164146CC8, VertexParallelism: 1, TaskAttemptID:attempt_1463524596314_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A699F028F1904A6ABC50C12C36A58900, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A6651F91CC08479EAF7C236D62BEDA32, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:36,501 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524596314_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:36,501 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:36,502 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:36,502 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:36,502 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:36,503 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524596314_0001_00_000001
2016-05-17 22:36:36,503 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524596314_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:36,504 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524596314_0001, with dag id: dag_1463524596314_0001_1
2016-05-17 22:36:36,506 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524596314_0001_00_000001
2016-05-17 22:36:36,506 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:36,506 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524596314_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:36,507 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596314_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524596314_0001_00_000001, launchTime=1463524596506
2016-05-17 22:36:36,506 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:36,508 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524596314_0001_00_000001
2016-05-17 22:36:36,509 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524596314_0001_1_00_000000_0] to container: [container_1463524596314_0001_00_000001]
2016-05-17 22:36:36,509 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524596314_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:36,510 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524596314_0001_00_000001 given task: attempt_1463524596314_0001_1_00_000000_0
2016-05-17 22:36:36,510 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524596314_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524596314_0001_1_00_000000_0
2016-05-17 22:36:36,510 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524596314_0001_1_00_000000_0] started. Is using containerId: [container_1463524596314_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:36,511 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:36,511 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596314_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E2B2EA6480344D69A84410A164146CC8, taskAttemptId=attempt_1463524596314_0001_1_00_000000_0, startTime=1463524596510, containerId=container_1463524596314_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524596314_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:36,512 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524596314_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:36 Starting to run new task attempt: attempt_1463524596314_0001_1_00_000000_0
2016-05-17 22:36:36,512 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524596314_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:36,512 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524596314_0001_1_00_000000_0
2016-05-17 22:36:36,513 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:36,513 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:36,513 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:36,514 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:36,514 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:36,514 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:36,514 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [CB3947E7F8AF4D84B754A061919FF90D/A0C0ABEF8E464D639B95A18FE56D7A7E] each/(1/1) ...mblyplatform/0_'field'_-1, VertexName: E2B2EA6480344D69A84410A164146CC8, VertexParallelism: 1, TaskAttemptID:attempt_1463524596314_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A699F028F1904A6ABC50C12C36A58900, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A6651F91CC08479EAF7C236D62BEDA32, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:36,514 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:36,515 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524596314_0001_1_00_000000_0
2016-05-17 22:36:36,515 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:36,516 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:36,516 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524596314_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:36,517 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A699F028F1904A6ABC50C12C36A58900, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:36,517 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A6651F91CC08479EAF7C236D62BEDA32, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:36,517 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:36,517 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:36,517 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:36,518 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A699F028F1904A6ABC50C12C36A58900
2016-05-17 22:36:36,518 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A699F028F1904A6ABC50C12C36A58900
2016-05-17 22:36:36,518 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A6651F91CC08479EAF7C236D62BEDA32
2016-05-17 22:36:36,518 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:36,519 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A6651F91CC08479EAF7C236D62BEDA32
2016-05-17 22:36:36,521 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:36,521 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A699F028F1904A6ABC50C12C36A58900
2016-05-17 22:36:36,522 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A699F028F1904A6ABC50C12C36A58900
2016-05-17 22:36:36,522 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:36,528 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E2B2EA6480344D69A84410A164146CC8, ordinal: 0
2016-05-17 22:36:36,528 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E2B2EA6480344D69A84410A164146CC8, mem on start (mb), free: 150, total: 388, max: 672
2016-05-17 22:36:36,529 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:36,529 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:36,529 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:36,530 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:36,530 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A6651F91CC08479EAF7C236D62BEDA32
2016-05-17 22:36:36,531 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:36,531 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:36,531 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:36,531 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:36,531 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:36,532 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:36,532 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:36,532 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:36,533 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A699F028F1904A6ABC50C12C36A58900, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:36,533 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A6651F91CC08479EAF7C236D62BEDA32, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:36,533 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:36,533 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A699F028F1904A6ABC50C12C36A58900
2016-05-17 22:36:36,533 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:36,533 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A699F028F1904A6ABC50C12C36A58900
2016-05-17 22:36:36,534 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A699F028F1904A6ABC50C12C36A58900 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:36,534 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:36,534 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:36,534 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:36,535 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524596314_0001_1_00_000000_0
2016-05-17 22:36:36,536 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:36,537 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:36,537 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@58bde4e2
2016-05-17 22:36:36,537 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: A699F028F1904A6ABC50C12C36A58900
2016-05-17 22:36:36,538 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_-1"], id: A6651F91CC08479EAF7C236D62BEDA32
2016-05-17 22:36:36,538 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_-1"]
2016-05-17 22:36:36,538 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:36,538 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:36,539 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@58bde4e2
2016-05-17 22:36:36,539 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:36,539 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:36,539 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E2B2EA6480344D69A84410A164146CC8, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:36,542 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:36,542 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:36,542 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524596314_0001_1_00_000000_0
2016-05-17 22:36:36,542 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524596314_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:36,543 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524596314_0001_1_00_000000_0
2016-05-17 22:36:36,543 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E2B2EA6480344D69A84410A164146CC8, mem on close (mb), free: 148, total: 388, max: 672
2016-05-17 22:36:36 Completed running task attempt: attempt_1463524596314_0001_1_00_000000_0
2016-05-17 22:36:36,544 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524596314_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:36,545 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524596540, Process_Duration=3, Process_End_Time=1463524596543, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:36,545 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:36,545 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596314_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E2B2EA6480344D69A84410A164146CC8, taskAttemptId=attempt_1463524596314_0001_1_00_000000_0, startTime=1463524596510, finishTime=1463524596545, timeTaken=35, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524596540, Process_Duration=3, Process_End_Time=1463524596543, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:36,546 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524596314_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:36,546 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:36,546 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524596314_0001_1_00_000000_0
2016-05-17 22:36:36,546 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:36,546 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:36,547 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524596314_0001_00_000001
2016-05-17 22:36:36,547 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596314_0001_1][Event:TASK_FINISHED]: vertexName=E2B2EA6480344D69A84410A164146CC8, taskId=task_1463524596314_0001_1_00_000000, startTime=1463524596510, finishTime=1463524596546, timeTaken=36, status=SUCCEEDED, successfulAttemptID=attempt_1463524596314_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524596540, Process_Duration=3, Process_End_Time=1463524596543, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:36,547 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524596314_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:36,547 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8] : 1
2016-05-17 22:36:36,547 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8]
2016-05-17 22:36:36,548 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:36,548 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:36,549 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596314_0001_1][Event:VERTEX_FINISHED]: vertexName=E2B2EA6480344D69A84410A164146CC8, vertexId=vertex_1463524596314_0001_1_00, initRequestedTime=1463524596486, initedTime=1463524596496, startRequestedTime=1463524596497, startedTime=1463524596497, finishTime=1463524596547, timeTaken=50, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524596540, Process_Duration=3, Process_End_Time=1463524596543, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524596510, firstTasksToStart=[ task_1463524596314_0001_1_00_000000 ], lastTaskFinishTime=1463524596545, lastTasksToFinish=[ task_1463524596314_0001_1_00_000000 ], minTaskDuration=35, maxTaskDuration=35, avgTaskDuration=35.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524596314_0001_1_00_000000 ], longestDurationTasks=[ task_1463524596314_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:36,549 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:36,549 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524596314_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:36,549 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596314_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524596314_0001_00_000001, stoppedTime=1463524596549, exitStatus=0
2016-05-17 22:36:36,549 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524596314_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:36,549 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:36,549 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524596314_0001_1
2016-05-17 22:36:36,549 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:36,549 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596314_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524596314_0001_1
2016-05-17 22:36:36,550 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: A6651F91CC08479EAF7C236D62BEDA32 for vertex: vertex_1463524596314_0001_1_00 [E2B2EA6480344D69A84410A164146CC8]
2016-05-17 22:36:36,557 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524596314_0001_1, queueSize=0
2016-05-17 22:36:36,558 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:36,558 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596314_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524596314_0001_1, startTime=1463524596485, finishTime=1463524596556, timeTaken=71, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=406847488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524596540, Process_Duration=3, Process_End_Time=1463524596543, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:36,558 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524596314_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:36,559 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524596314_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:36,559 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524596314_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:36,559 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524596314_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:36:36,559 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524596314_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:36,559 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524596314_0001_00_000001, Exiting
2016-05-17 22:36:36,559 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524596314_0001_00_000001
2016-05-17 22:36:36,560 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524596314_0001_00_000001
2016-05-17 22:36:36,561 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524596314_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:36 Completed Dag: dag_1463524596314_0001_1
2016-05-17 22:36:36,561 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524596314_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:36,561 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524596314_0001_1_post
2016-05-17 22:36:36,561 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:36,562 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:36,561 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524596314_0001_00_000001 completed successfully
2016-05-17 22:36:36,562 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524596314_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:36:36,563 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524596314_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:36:36,566 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...mblyplatform/0_'field'_-1, applicationId=application_1463524596314_0001
2016-05-17 22:36:36,568 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:36,568 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:36,568 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:36,568 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:36,568 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:36,569 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:36,570 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:36,570 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:36,570 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:36,570 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:36,571 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:36,571 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:36,571 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:36,576 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:36,576 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:36,577 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CB3947E7F8AF4D84B754A061919FF90D/A0C0ABEF8E464D639B95A18FE56D7A7E/.tez/application_1463524596314_0001
2016-05-17 22:36:36,577 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_-1/_temporary
2016-05-17 22:36:36,578 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.282
2016-05-17 22:36:36,658 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_'field'_-1
2016-05-17 22:36:36,680 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:36,682 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:36,713 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-05-17 22:36:36,718 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 22:36:36,718 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:36,719 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:36,719 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:36,733 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:36,734 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_'field'
2016-05-17 22:36:36,745 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:36,745 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:36,746 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_'field'"]
2016-05-17 22:36:36,746 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:36,746 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:36,746 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:36,747 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...latform/0_'field'_'field'
2016-05-17 22:36:36,747 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:36,759 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:36,763 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:36,764 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:36,764 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:36,773 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E15D4E97D398403B9A020BF4CDE97109/D914BFBB2C3F4CCAB81081FE4775C646/.tez/application_1463524596764_0001 doesn't exist and is created
2016-05-17 22:36:36,789 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:36,789 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:36,790 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E15D4E97D398403B9A020BF4CDE97109/D914BFBB2C3F4CCAB81081FE4775C646/.tez/application_1463524596764_0001
2016-05-17 22:36:36,793 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524596764_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:36,803 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:36,803 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:36,804 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:36,805 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:36,805 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E15D4E97D398403B9A020BF4CDE97109/D914BFBB2C3F4CCAB81081FE4775C646/.tez/application_1463524596764_0001/localmode-log-dir
2016-05-17 22:36:36,805 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E15D4E97D398403B9A020BF4CDE97109/D914BFBB2C3F4CCAB81081FE4775C646/.tez/application_1463524596764_0001/localmode-log-dir/history.txt.appattempt_1463524596764_0001_000000, maxErrors=10
2016-05-17 22:36:36,805 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:36,805 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524596764_0001_000000, appSubmitTime=1463524596792, launchTime=1463524596793
2016-05-17 22:36:36,809 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:36,810 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:36,811 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:36,812 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:36,820 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:35610
2016-05-17 22:36:36,823 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:36,826 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524596764_0001_000000, startTime=1463524596825
2016-05-17 22:36:36,826 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:36,826 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:36,890 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:36,890 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:36,890 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/0_'field'_'field', applicationId=application_1463524596764_0001, dagName=[E15D4E97D398403B9A020BF4CDE97109/D914BFBB2C3F4CCAB81081FE4775C646] each/(1/1) ...latform/0_'field'_'field'
2016-05-17 22:36:36,905 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [E15D4E97D398403B9A020BF4CDE97109/D914BFBB2C3F4CCAB81081FE4775C646] each/(1/1) ...latform/0_'field'_'field'
2016-05-17 22:36:36,906 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524596764_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E15D4E97D398403B9A020BF4CDE97109/D914BFBB2C3F4CCAB81081FE4775C646/.tez/application_1463524596764_0001/localmode-log-dir/dag_1463524596764_0001_1.dot
2016-05-17 22:36:36,907 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E15D4E97D398403B9A020BF4CDE97109/D914BFBB2C3F4CCAB81081FE4775C646/.tez/application_1463524596764_0001/localmode-log-dir/dag_1463524596764_0001_1-tez-dag.pb.txt
2016-05-17 22:36:36,911 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524596764_0001_1
2016-05-17 22:36:36,911 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:36,911 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [E15D4E97D398403B9A020BF4CDE97109/D914BFBB2C3F4CCAB81081FE4775C646] each/(1/1) ...latform/0_'field'_'field'
2016-05-17 22:36:36 Running Dag: dag_1463524596764_0001_1
2016-05-17 22:36:36,923 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596764_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524596764_0001_1, submitTime=1463524596905
2016-05-17 22:36:36,923 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:36,923 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D6AF8A021FEC4038B3EA097425789369
2016-05-17 22:36:36,923 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:36,924 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596764_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524596764_0001_1, initTime=1463524596923
2016-05-17 22:36:36,924 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:36,925 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524596764_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:36,927 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596764_0001_1][Event:DAG_STARTED]: dagID=dag_1463524596764_0001_1, startTime=1463524596925
2016-05-17 22:36:36,927 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:36,927 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:36,927 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524596764_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:36,927 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D6AF8A021FEC4038B3EA097425789369 : {22AF8F3E90684AB99465D172A76D530F={InputName=22AF8F3E90684AB99465D172A76D530F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:36,927 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 22AF8F3E90684AB99465D172A76D530F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:36,928 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369]
2016-05-17 22:36:36,928 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369]
2016-05-17 22:36:36,928 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369]
2016-05-17 22:36:36,928 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369]. Starting root input initializers: 1
2016-05-17 22:36:36,928 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/0_'field'_'field', applicationId=application_1463524596764_0001, dagName=[E15D4E97D398403B9A020BF4CDE97109/D914BFBB2C3F4CCAB81081FE4775C646] each/(1/1) ...latform/0_'field'_'field'
2016-05-17 22:36:36,929 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:36,929 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 22AF8F3E90684AB99465D172A76D530F on vertex vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369]
2016-05-17 22:36:36,930 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:36,931 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 22AF8F3E90684AB99465D172A76D530F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:36,931 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:36,931 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:36,933 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:36,933 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 22AF8F3E90684AB99465D172A76D530F on vertex vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369]
2016-05-17 22:36:36,933 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:36,933 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {22AF8F3E90684AB99465D172A76D530F=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:36,933 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369] parallelism set to 1
2016-05-17 22:36:36,933 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:36,934 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596764_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524596764_0001_1_00, eventCount=1
2016-05-17 22:36:36,934 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369]
2016-05-17 22:36:36,934 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=EC31BEEDDF8B463894CE26AB8DC093CD, vertexId=vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:36,934 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=EC31BEEDDF8B463894CE26AB8DC093CD, vertexId=vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369]
2016-05-17 22:36:36,936 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=EC31BEEDDF8B463894CE26AB8DC093CD, vertexName=D6AF8A021FEC4038B3EA097425789369, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:36,936 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=EC31BEEDDF8B463894CE26AB8DC093CD, vertexId=vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369]
2016-05-17 22:36:36,937 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:36,937 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596764_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D6AF8A021FEC4038B3EA097425789369, vertexId=vertex_1463524596764_0001_1_00, initRequestedTime=1463524596927, initedTime=1463524596937, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:36,937 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369] with distanceFromRoot: 0
2016-05-17 22:36:36,937 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:36,937 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D6AF8A021FEC4038B3EA097425789369
2016-05-17 22:36:36,937 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369]
2016-05-17 22:36:36,938 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:36,938 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596764_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524596764_0001_1_00, startRequestedTime=1463524596937, startedTime=1463524596937
2016-05-17 22:36:36,938 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:36,938 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:36,939 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596764_0001_1][Event:TASK_STARTED]: vertexName=D6AF8A021FEC4038B3EA097425789369, taskId=task_1463524596764_0001_1_00_000000, scheduledTime=1463524596938, launchTime=1463524596938
2016-05-17 22:36:36,939 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524596764_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:36,939 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369]
2016-05-17 22:36:36,942 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E15D4E97D398403B9A020BF4CDE97109/D914BFBB2C3F4CCAB81081FE4775C646] each/(1/1) ...latform/0_'field'_'field', VertexName: D6AF8A021FEC4038B3EA097425789369, VertexParallelism: 1, TaskAttemptID:attempt_1463524596764_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=22AF8F3E90684AB99465D172A76D530F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EC31BEEDDF8B463894CE26AB8DC093CD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:36,942 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524596764_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:36,943 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:36,943 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:36,943 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:36,944 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:36,944 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524596764_0001_00_000001
2016-05-17 22:36:36,944 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524596764_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:36,946 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524596764_0001_00_000001
2016-05-17 22:36:36,946 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:36,946 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524596764_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:36,947 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524596764_0001, with dag id: dag_1463524596764_0001_1
2016-05-17 22:36:36,947 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:36,947 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596764_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524596764_0001_00_000001, launchTime=1463524596946
2016-05-17 22:36:36,948 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524596764_0001_00_000001
2016-05-17 22:36:36,948 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524596764_0001_1_00_000000_0] to container: [container_1463524596764_0001_00_000001]
2016-05-17 22:36:36,948 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524596764_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:36,949 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524596764_0001_00_000001 given task: attempt_1463524596764_0001_1_00_000000_0
2016-05-17 22:36:36,949 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524596764_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524596764_0001_1_00_000000_0
2016-05-17 22:36:36,949 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524596764_0001_1_00_000000_0] started. Is using containerId: [container_1463524596764_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:36,949 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596764_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D6AF8A021FEC4038B3EA097425789369, taskAttemptId=attempt_1463524596764_0001_1_00_000000_0, startTime=1463524596949, containerId=container_1463524596764_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524596764_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:36 Starting to run new task attempt: attempt_1463524596764_0001_1_00_000000_0
2016-05-17 22:36:36,949 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:36,949 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524596764_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:36,949 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524596764_0001_1_00_000000_0
2016-05-17 22:36:36,950 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:36,950 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524596764_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:36,950 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:36,950 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:36,950 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:36,951 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:36,951 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:36,952 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E15D4E97D398403B9A020BF4CDE97109/D914BFBB2C3F4CCAB81081FE4775C646] each/(1/1) ...latform/0_'field'_'field', VertexName: D6AF8A021FEC4038B3EA097425789369, VertexParallelism: 1, TaskAttemptID:attempt_1463524596764_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=22AF8F3E90684AB99465D172A76D530F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EC31BEEDDF8B463894CE26AB8DC093CD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:36,952 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:36,952 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524596764_0001_1_00_000000_0
2016-05-17 22:36:36,952 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:36,952 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:36,953 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:36,954 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EC31BEEDDF8B463894CE26AB8DC093CD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:36,955 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:36,955 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524596764_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:36,955 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EC31BEEDDF8B463894CE26AB8DC093CD
2016-05-17 22:36:36,955 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:36,956 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: EC31BEEDDF8B463894CE26AB8DC093CD
2016-05-17 22:36:36,956 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:36,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=22AF8F3E90684AB99465D172A76D530F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:36,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:36,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 22AF8F3E90684AB99465D172A76D530F
2016-05-17 22:36:36,959 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 22AF8F3E90684AB99465D172A76D530F
2016-05-17 22:36:36,964 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:36,964 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EC31BEEDDF8B463894CE26AB8DC093CD
2016-05-17 22:36:36,965 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:36,965 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D6AF8A021FEC4038B3EA097425789369, ordinal: 0
2016-05-17 22:36:36,965 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D6AF8A021FEC4038B3EA097425789369, mem on start (mb), free: 285, total: 389, max: 672
2016-05-17 22:36:36,965 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:36,965 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:36,965 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:36,965 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 22AF8F3E90684AB99465D172A76D530F
2016-05-17 22:36:36,966 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 22AF8F3E90684AB99465D172A76D530F
2016-05-17 22:36:36,966 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:36,966 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:36,966 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:36,966 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:36,966 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:36,966 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:36,967 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:36,967 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:36,967 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EC31BEEDDF8B463894CE26AB8DC093CD, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:36,967 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 22AF8F3E90684AB99465D172A76D530F, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:36,967 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:36,967 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 22AF8F3E90684AB99465D172A76D530F being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:36,967 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 22AF8F3E90684AB99465D172A76D530F
2016-05-17 22:36:36,967 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:36,968 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 22AF8F3E90684AB99465D172A76D530F
2016-05-17 22:36:36,967 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:36,968 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:36,968 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:36,968 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524596764_0001_1_00_000000_0
2016-05-17 22:36:36,968 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:36,969 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@39986001
2016-05-17 22:36:36,971 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@39986001
2016-05-17 22:36:36,971 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:36,971 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:36,972 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:36,972 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 22AF8F3E90684AB99465D172A76D530F
2016-05-17 22:36:36,972 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_'field'"], id: EC31BEEDDF8B463894CE26AB8DC093CD
2016-05-17 22:36:36,973 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_'field'"]
2016-05-17 22:36:36,973 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:36,973 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:36,973 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D6AF8A021FEC4038B3EA097425789369, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:36,975 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:36,975 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:36,976 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524596764_0001_1_00_000000_0
2016-05-17 22:36:36,976 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524596764_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:36,976 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524596764_0001_1_00_000000_0
2016-05-17 22:36:36,977 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D6AF8A021FEC4038B3EA097425789369, mem on close (mb), free: 283, total: 389, max: 672
2016-05-17 22:36:36 Completed running task attempt: attempt_1463524596764_0001_1_00_000000_0
2016-05-17 22:36:36,977 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524596764_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:36,979 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407896064, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524596973, Process_Duration=3, Process_End_Time=1463524596976, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:36,979 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:36,979 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596764_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D6AF8A021FEC4038B3EA097425789369, taskAttemptId=attempt_1463524596764_0001_1_00_000000_0, startTime=1463524596949, finishTime=1463524596978, timeTaken=29, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407896064, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524596973, Process_Duration=3, Process_End_Time=1463524596976, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:36,979 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524596764_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:36,979 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:36,980 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:36,979 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:36,980 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524596764_0001_1_00_000000_0
2016-05-17 22:36:36,982 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596764_0001_1][Event:TASK_FINISHED]: vertexName=D6AF8A021FEC4038B3EA097425789369, taskId=task_1463524596764_0001_1_00_000000, startTime=1463524596949, finishTime=1463524596981, timeTaken=32, status=SUCCEEDED, successfulAttemptID=attempt_1463524596764_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407896064, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524596973, Process_Duration=3, Process_End_Time=1463524596976, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:36,982 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524596764_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:36,982 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369] : 1
2016-05-17 22:36:36,982 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369]
2016-05-17 22:36:36,983 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524596764_0001_00_000001
2016-05-17 22:36:36,983 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596764_0001_1][Event:VERTEX_FINISHED]: vertexName=D6AF8A021FEC4038B3EA097425789369, vertexId=vertex_1463524596764_0001_1_00, initRequestedTime=1463524596927, initedTime=1463524596937, startRequestedTime=1463524596937, startedTime=1463524596937, finishTime=1463524596982, timeTaken=45, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407896064, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524596973, Process_Duration=3, Process_End_Time=1463524596976, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524596949, firstTasksToStart=[ task_1463524596764_0001_1_00_000000 ], lastTaskFinishTime=1463524596978, lastTasksToFinish=[ task_1463524596764_0001_1_00_000000 ], minTaskDuration=29, maxTaskDuration=29, avgTaskDuration=29.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524596764_0001_1_00_000000 ], longestDurationTasks=[ task_1463524596764_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:36,982 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:36,984 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:36,984 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524596764_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:36,984 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596764_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524596764_0001_00_000001, stoppedTime=1463524596984, exitStatus=0
2016-05-17 22:36:36,984 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:36,984 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524596764_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:36,984 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:36,985 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524596764_0001_1
2016-05-17 22:36:36,985 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596764_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524596764_0001_1
2016-05-17 22:36:36,985 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:36,985 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: EC31BEEDDF8B463894CE26AB8DC093CD for vertex: vertex_1463524596764_0001_1_00 [D6AF8A021FEC4038B3EA097425789369]
2016-05-17 22:36:36,993 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524596764_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:36,994 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524596764_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:36,994 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524596764_0001_00_000001, Exiting
2016-05-17 22:36:36,994 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524596764_0001_00_000001
2016-05-17 22:36:36,994 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524596764_0001_00_000001
2016-05-17 22:36:36,995 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524596764_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:36,996 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524596764_0001_00_000001 completed successfully
2016-05-17 22:36:36,997 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524596764_0001_1, queueSize=0
2016-05-17 22:36:36,997 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:36,998 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524596764_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524596764_0001_1, startTime=1463524596925, finishTime=1463524596994, timeTaken=69, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407896064, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524596973, Process_Duration=3, Process_End_Time=1463524596976, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:36,998 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524596764_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:36,998 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524596764_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:36,998 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524596764_0001_00_000001
2016-05-17 22:36:36,998 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524596764_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:36 Completed Dag: dag_1463524596764_0001_1
2016-05-17 22:36:36,999 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524596764_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:36,999 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524596764_0001_1_post
2016-05-17 22:36:36,999 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:37,000 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:37,004 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/0_'field'_'field', applicationId=application_1463524596764_0001
2016-05-17 22:36:37,005 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:37,005 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:37,006 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:37,006 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:37,006 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:37,007 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:37,007 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:37,007 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:37,007 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:37,009 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:37,009 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:37,009 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:37,009 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:37,009 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:37,009 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:37,014 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E15D4E97D398403B9A020BF4CDE97109/D914BFBB2C3F4CCAB81081FE4775C646/.tez/application_1463524596764_0001
2016-05-17 22:36:37,014 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_'field'/_temporary
2016-05-17 22:36:37,015 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.269
2016-05-17 22:36:37,080 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_'field'_'field'
2016-05-17 22:36:37,100 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:37,101 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:37,129 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 22:36:37,136 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 22:36:37,137 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:37,137 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:37,137 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:37,152 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:37,153 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_RESULTS
2016-05-17 22:36:37,162 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:37,163 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:37,163 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_RESULTS"]
2016-05-17 22:36:37,163 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:37,163 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:37,163 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:37,164 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...latform/0_'field'_RESULTS
2016-05-17 22:36:37,164 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:37,176 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:37,180 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:37,180 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:37,180 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:37,189 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/86CA0625588742FABA0B87C58582B123/FCD61315DF064585B47BF20B1219E3C9/.tez/application_1463524597180_0001 doesn't exist and is created
2016-05-17 22:36:37,205 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:37,205 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:37,205 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/86CA0625588742FABA0B87C58582B123/FCD61315DF064585B47BF20B1219E3C9/.tez/application_1463524597180_0001
2016-05-17 22:36:37,208 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524597180_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:37,219 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:37,219 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:37,221 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:37,221 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:37,221 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/86CA0625588742FABA0B87C58582B123/FCD61315DF064585B47BF20B1219E3C9/.tez/application_1463524597180_0001/localmode-log-dir
2016-05-17 22:36:37,222 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/86CA0625588742FABA0B87C58582B123/FCD61315DF064585B47BF20B1219E3C9/.tez/application_1463524597180_0001/localmode-log-dir/history.txt.appattempt_1463524597180_0001_000000, maxErrors=10
2016-05-17 22:36:37,222 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:37,222 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524597180_0001_000000, appSubmitTime=1463524597208, launchTime=1463524597208
2016-05-17 22:36:37,227 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:43732
2016-05-17 22:36:37,228 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:37,230 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:37,231 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:37,231 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:37,232 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:37,234 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:37,234 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524597180_0001_000000, startTime=1463524597234
2016-05-17 22:36:37,234 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:37,305 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:37,305 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:37,305 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/0_'field'_RESULTS, applicationId=application_1463524597180_0001, dagName=[86CA0625588742FABA0B87C58582B123/FCD61315DF064585B47BF20B1219E3C9] each/(1/1) ...latform/0_'field'_RESULTS
2016-05-17 22:36:37,320 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [86CA0625588742FABA0B87C58582B123/FCD61315DF064585B47BF20B1219E3C9] each/(1/1) ...latform/0_'field'_RESULTS
2016-05-17 22:36:37,321 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524597180_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/86CA0625588742FABA0B87C58582B123/FCD61315DF064585B47BF20B1219E3C9/.tez/application_1463524597180_0001/localmode-log-dir/dag_1463524597180_0001_1.dot
2016-05-17 22:36:37,321 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/86CA0625588742FABA0B87C58582B123/FCD61315DF064585B47BF20B1219E3C9/.tez/application_1463524597180_0001/localmode-log-dir/dag_1463524597180_0001_1-tez-dag.pb.txt
2016-05-17 22:36:37,325 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524597180_0001_1
2016-05-17 22:36:37,325 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:37,325 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [86CA0625588742FABA0B87C58582B123/FCD61315DF064585B47BF20B1219E3C9] each/(1/1) ...latform/0_'field'_RESULTS
2016-05-17 22:36:37 Running Dag: dag_1463524597180_0001_1
2016-05-17 22:36:37,338 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597180_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524597180_0001_1, submitTime=1463524597320
2016-05-17 22:36:37,339 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:37,339 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BEBA9C971C8B4C8FADEEDC0F005910F7
2016-05-17 22:36:37,339 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:37,340 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597180_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524597180_0001_1, initTime=1463524597339
2016-05-17 22:36:37,340 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524597180_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:37,340 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:37,341 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597180_0001_1][Event:DAG_STARTED]: dagID=dag_1463524597180_0001_1, startTime=1463524597341
2016-05-17 22:36:37,341 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:37,341 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524597180_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:37,341 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:37,341 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BEBA9C971C8B4C8FADEEDC0F005910F7 : {217475CDA1B148CE941C5EF321050190={InputName=217475CDA1B148CE941C5EF321050190}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:37,341 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 217475CDA1B148CE941C5EF321050190, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:37,342 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7]
2016-05-17 22:36:37,342 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7]
2016-05-17 22:36:37,342 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7]
2016-05-17 22:36:37,342 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7]. Starting root input initializers: 1
2016-05-17 22:36:37,342 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/0_'field'_RESULTS, applicationId=application_1463524597180_0001, dagName=[86CA0625588742FABA0B87C58582B123/FCD61315DF064585B47BF20B1219E3C9] each/(1/1) ...latform/0_'field'_RESULTS
2016-05-17 22:36:37,342 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:37,343 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 217475CDA1B148CE941C5EF321050190 on vertex vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7]
2016-05-17 22:36:37,345 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:37,345 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 217475CDA1B148CE941C5EF321050190 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:37,345 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:37,346 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:37,347 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:37,347 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 217475CDA1B148CE941C5EF321050190 on vertex vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7]
2016-05-17 22:36:37,348 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:37,348 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {217475CDA1B148CE941C5EF321050190=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:37,348 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7] parallelism set to 1
2016-05-17 22:36:37,348 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:37,348 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597180_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524597180_0001_1_00, eventCount=1
2016-05-17 22:36:37,348 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7]
2016-05-17 22:36:37,348 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D559DB3935024D779B28E1566C54A546, vertexId=vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:37,349 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D559DB3935024D779B28E1566C54A546, vertexId=vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7]
2016-05-17 22:36:37,350 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D559DB3935024D779B28E1566C54A546, vertexName=BEBA9C971C8B4C8FADEEDC0F005910F7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:37,350 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D559DB3935024D779B28E1566C54A546, vertexId=vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7]
2016-05-17 22:36:37,351 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597180_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BEBA9C971C8B4C8FADEEDC0F005910F7, vertexId=vertex_1463524597180_0001_1_00, initRequestedTime=1463524597341, initedTime=1463524597351, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:37,351 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7] with distanceFromRoot: 0
2016-05-17 22:36:37,352 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:37,352 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:37,352 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BEBA9C971C8B4C8FADEEDC0F005910F7
2016-05-17 22:36:37,352 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7]
2016-05-17 22:36:37,353 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597180_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524597180_0001_1_00, startRequestedTime=1463524597352, startedTime=1463524597352
2016-05-17 22:36:37,353 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:37,353 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:37,354 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597180_0001_1][Event:TASK_STARTED]: vertexName=BEBA9C971C8B4C8FADEEDC0F005910F7, taskId=task_1463524597180_0001_1_00_000000, scheduledTime=1463524597353, launchTime=1463524597353
2016-05-17 22:36:37,354 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:37,354 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524597180_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:37,354 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7]
2016-05-17 22:36:37,355 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [86CA0625588742FABA0B87C58582B123/FCD61315DF064585B47BF20B1219E3C9] each/(1/1) ...latform/0_'field'_RESULTS, VertexName: BEBA9C971C8B4C8FADEEDC0F005910F7, VertexParallelism: 1, TaskAttemptID:attempt_1463524597180_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=217475CDA1B148CE941C5EF321050190, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D559DB3935024D779B28E1566C54A546, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:37,355 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524597180_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:37,355 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:37,356 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:37,356 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:37,356 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:37,357 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524597180_0001_00_000001
2016-05-17 22:36:37,357 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524597180_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:37,358 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524597180_0001_00_000001
2016-05-17 22:36:37,358 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:37,359 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524597180_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:37,359 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524597180_0001, with dag id: dag_1463524597180_0001_1
2016-05-17 22:36:37,361 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:37,362 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597180_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524597180_0001_00_000001, launchTime=1463524597359
2016-05-17 22:36:37,362 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524597180_0001_00_000001
2016-05-17 22:36:37,363 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524597180_0001_1_00_000000_0] to container: [container_1463524597180_0001_00_000001]
2016-05-17 22:36:37,363 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524597180_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:37,363 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524597180_0001_1_00_000000_0] started. Is using containerId: [container_1463524597180_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:37,364 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597180_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BEBA9C971C8B4C8FADEEDC0F005910F7, taskAttemptId=attempt_1463524597180_0001_1_00_000000_0, startTime=1463524597363, containerId=container_1463524597180_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524597180_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:37,364 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524597180_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:37,364 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524597180_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:37,364 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:37,364 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524597180_0001_00_000001 given task: attempt_1463524597180_0001_1_00_000000_0
2016-05-17 22:36:37,365 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524597180_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524597180_0001_1_00_000000_0
2016-05-17 22:36:37 Starting to run new task attempt: attempt_1463524597180_0001_1_00_000000_0
2016-05-17 22:36:37,365 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524597180_0001_1_00_000000_0
2016-05-17 22:36:37,365 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:37,365 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:37,365 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:37,366 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:37,366 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:37,366 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:37,366 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [86CA0625588742FABA0B87C58582B123/FCD61315DF064585B47BF20B1219E3C9] each/(1/1) ...latform/0_'field'_RESULTS, VertexName: BEBA9C971C8B4C8FADEEDC0F005910F7, VertexParallelism: 1, TaskAttemptID:attempt_1463524597180_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=217475CDA1B148CE941C5EF321050190, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D559DB3935024D779B28E1566C54A546, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:37,366 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:37,367 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524597180_0001_1_00_000000_0
2016-05-17 22:36:37,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:37,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:37,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=217475CDA1B148CE941C5EF321050190, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:37,370 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D559DB3935024D779B28E1566C54A546, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:37,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:37,370 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:37,370 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524597180_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:37,372 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:37,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 217475CDA1B148CE941C5EF321050190
2016-05-17 22:36:37,374 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 217475CDA1B148CE941C5EF321050190
2016-05-17 22:36:37,374 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:37,375 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D559DB3935024D779B28E1566C54A546
2016-05-17 22:36:37,375 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:37,375 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D559DB3935024D779B28E1566C54A546
2016-05-17 22:36:37,379 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:37,379 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 217475CDA1B148CE941C5EF321050190
2016-05-17 22:36:37,380 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 217475CDA1B148CE941C5EF321050190
2016-05-17 22:36:37,382 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BEBA9C971C8B4C8FADEEDC0F005910F7, ordinal: 0
2016-05-17 22:36:37,382 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BEBA9C971C8B4C8FADEEDC0F005910F7, mem on start (mb), free: 208, total: 389, max: 672
2016-05-17 22:36:37,382 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:37,382 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:37,382 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:37,387 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:37,387 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D559DB3935024D779B28E1566C54A546
2016-05-17 22:36:37,387 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:37,387 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:37,387 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:37,387 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:37,387 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:37,388 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:37,388 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:37,388 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:37,388 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 217475CDA1B148CE941C5EF321050190, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:37,388 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D559DB3935024D779B28E1566C54A546, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:37,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:37,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 217475CDA1B148CE941C5EF321050190 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:37,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:37,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:37,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 217475CDA1B148CE941C5EF321050190
2016-05-17 22:36:37,389 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:37,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 217475CDA1B148CE941C5EF321050190
2016-05-17 22:36:37,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:37,389 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524597180_0001_1_00_000000_0
2016-05-17 22:36:37,389 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:37,392 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:37,392 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 217475CDA1B148CE941C5EF321050190
2016-05-17 22:36:37,393 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_RESULTS"], id: D559DB3935024D779B28E1566C54A546
2016-05-17 22:36:37,393 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_RESULTS"]
2016-05-17 22:36:37,393 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:37,393 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:37,396 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@59909ac7
2016-05-17 22:36:37,397 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@59909ac7
2016-05-17 22:36:37,397 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:37,397 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:37,397 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BEBA9C971C8B4C8FADEEDC0F005910F7, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:37,400 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:37,400 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:37,400 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524597180_0001_1_00_000000_0
2016-05-17 22:36:37,401 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524597180_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:37,401 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524597180_0001_1_00_000000_0
2016-05-17 22:36:37,401 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BEBA9C971C8B4C8FADEEDC0F005910F7, mem on close (mb), free: 206, total: 389, max: 672
2016-05-17 22:36:37 Completed running task attempt: attempt_1463524597180_0001_1_00_000000_0
2016-05-17 22:36:37,402 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524597180_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:37,404 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407896064, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524597397, Process_Duration=4, Process_End_Time=1463524597401, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:37,404 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597180_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BEBA9C971C8B4C8FADEEDC0F005910F7, taskAttemptId=attempt_1463524597180_0001_1_00_000000_0, startTime=1463524597363, finishTime=1463524597404, timeTaken=41, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407896064, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524597397, Process_Duration=4, Process_End_Time=1463524597401, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:37,404 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:37,404 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524597180_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:37,404 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524597180_0001_1_00_000000_0
2016-05-17 22:36:37,404 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:37,405 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:37,406 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524597180_0001_00_000001
2016-05-17 22:36:37,407 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:37,408 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:37,408 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597180_0001_1][Event:TASK_FINISHED]: vertexName=BEBA9C971C8B4C8FADEEDC0F005910F7, taskId=task_1463524597180_0001_1_00_000000, startTime=1463524597363, finishTime=1463524597404, timeTaken=41, status=SUCCEEDED, successfulAttemptID=attempt_1463524597180_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407896064, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524597397, Process_Duration=4, Process_End_Time=1463524597401, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:37,408 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524597180_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:37,408 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7] : 1
2016-05-17 22:36:37,408 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7]
2016-05-17 22:36:37,409 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:37,409 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597180_0001_1][Event:VERTEX_FINISHED]: vertexName=BEBA9C971C8B4C8FADEEDC0F005910F7, vertexId=vertex_1463524597180_0001_1_00, initRequestedTime=1463524597341, initedTime=1463524597351, startRequestedTime=1463524597352, startedTime=1463524597352, finishTime=1463524597408, timeTaken=56, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407896064, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524597397, Process_Duration=4, Process_End_Time=1463524597401, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524597363, firstTasksToStart=[ task_1463524597180_0001_1_00_000000 ], lastTaskFinishTime=1463524597404, lastTasksToFinish=[ task_1463524597180_0001_1_00_000000 ], minTaskDuration=41, maxTaskDuration=41, avgTaskDuration=41.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524597180_0001_1_00_000000 ], longestDurationTasks=[ task_1463524597180_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:37,410 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:37,410 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524597180_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:37,410 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597180_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524597180_0001_00_000001, stoppedTime=1463524597410, exitStatus=0
2016-05-17 22:36:37,410 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:37,410 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524597180_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:37,410 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:37,410 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524597180_0001_1
2016-05-17 22:36:37,411 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597180_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524597180_0001_1
2016-05-17 22:36:37,411 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: D559DB3935024D779B28E1566C54A546 for vertex: vertex_1463524597180_0001_1_00 [BEBA9C971C8B4C8FADEEDC0F005910F7]
2016-05-17 22:36:37,417 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524597180_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:37,417 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524597180_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:37,417 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524597180_0001_00_000001, Exiting
2016-05-17 22:36:37,417 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524597180_0001_00_000001
2016-05-17 22:36:37,418 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524597180_0001_00_000001
2016-05-17 22:36:37,420 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524597180_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:37,421 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524597180_0001_00_000001 completed successfully
2016-05-17 22:36:37,422 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524597180_0001_1, queueSize=0
2016-05-17 22:36:37,422 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:37,422 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597180_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524597180_0001_1, startTime=1463524597341, finishTime=1463524597420, timeTaken=79, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407896064, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524597397, Process_Duration=4, Process_End_Time=1463524597401, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:37,422 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524597180_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:37,422 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524597180_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:37,422 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524597180_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:37 Completed Dag: dag_1463524597180_0001_1
2016-05-17 22:36:37,423 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524597180_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:37,423 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524597180_0001_1_post
2016-05-17 22:36:37,423 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:37,423 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:37,423 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524597180_0001_00_000001
2016-05-17 22:36:37,429 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/0_'field'_RESULTS, applicationId=application_1463524597180_0001
2016-05-17 22:36:37,431 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:37,431 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:37,432 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:37,432 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:37,432 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:37,433 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:37,433 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:37,434 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:37,434 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:37,434 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:37,435 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:37,435 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:37,436 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:37,440 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/86CA0625588742FABA0B87C58582B123/FCD61315DF064585B47BF20B1219E3C9/.tez/application_1463524597180_0001
2016-05-17 22:36:37,441 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_RESULTS/_temporary
2016-05-17 22:36:37,441 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.278
2016-05-17 22:36:37,508 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:37,508 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:37,509 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_'field'_RESULTS
2016-05-17 22:36:37,530 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:37,531 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:37,564 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 22:36:37,570 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-17 22:36:37,570 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:37,570 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:37,571 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:37,584 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:37,585 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_ALL
2016-05-17 22:36:37,593 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:37,594 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:37,594 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_ALL"]
2016-05-17 22:36:37,595 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:37,595 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:37,596 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:37,597 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...blyplatform/0_'field'_ALL
2016-05-17 22:36:37,598 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:37,613 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:37,618 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:37,618 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:37,618 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:37,627 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D1511BC5B7F34357B4556137389C072C/D75AABD94B3E4B21B818DC3DA7957B83/.tez/application_1463524597618_0001 doesn't exist and is created
2016-05-17 22:36:37,642 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:37,642 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:37,643 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D1511BC5B7F34357B4556137389C072C/D75AABD94B3E4B21B818DC3DA7957B83/.tez/application_1463524597618_0001
2016-05-17 22:36:37,645 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524597618_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:37,655 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:37,655 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:37,657 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:37,658 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:37,658 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D1511BC5B7F34357B4556137389C072C/D75AABD94B3E4B21B818DC3DA7957B83/.tez/application_1463524597618_0001/localmode-log-dir
2016-05-17 22:36:37,658 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D1511BC5B7F34357B4556137389C072C/D75AABD94B3E4B21B818DC3DA7957B83/.tez/application_1463524597618_0001/localmode-log-dir/history.txt.appattempt_1463524597618_0001_000000, maxErrors=10
2016-05-17 22:36:37,659 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:37,659 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524597618_0001_000000, appSubmitTime=1463524597645, launchTime=1463524597645
2016-05-17 22:36:37,662 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:37,665 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:37,665 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:37,667 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:37,669 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:40376
2016-05-17 22:36:37,670 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:37,672 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524597618_0001_000000, startTime=1463524597672
2016-05-17 22:36:37,672 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:37,672 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:37,743 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:37,743 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:37,743 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/0_'field'_ALL, applicationId=application_1463524597618_0001, dagName=[D1511BC5B7F34357B4556137389C072C/D75AABD94B3E4B21B818DC3DA7957B83] each/(1/1) ...blyplatform/0_'field'_ALL
2016-05-17 22:36:37,757 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [D1511BC5B7F34357B4556137389C072C/D75AABD94B3E4B21B818DC3DA7957B83] each/(1/1) ...blyplatform/0_'field'_ALL
2016-05-17 22:36:37,758 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524597618_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D1511BC5B7F34357B4556137389C072C/D75AABD94B3E4B21B818DC3DA7957B83/.tez/application_1463524597618_0001/localmode-log-dir/dag_1463524597618_0001_1.dot
2016-05-17 22:36:37,759 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D1511BC5B7F34357B4556137389C072C/D75AABD94B3E4B21B818DC3DA7957B83/.tez/application_1463524597618_0001/localmode-log-dir/dag_1463524597618_0001_1-tez-dag.pb.txt
2016-05-17 22:36:37,763 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524597618_0001_1
2016-05-17 22:36:37,763 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:37,763 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [D1511BC5B7F34357B4556137389C072C/D75AABD94B3E4B21B818DC3DA7957B83] each/(1/1) ...blyplatform/0_'field'_ALL
2016-05-17 22:36:37 Running Dag: dag_1463524597618_0001_1
2016-05-17 22:36:37,775 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597618_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524597618_0001_1, submitTime=1463524597757
2016-05-17 22:36:37,775 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:37,775 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D3B4F6252A7747918D2D64D018FAC446
2016-05-17 22:36:37,776 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:37,776 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597618_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524597618_0001_1, initTime=1463524597775
2016-05-17 22:36:37,776 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:37,776 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524597618_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:37,777 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597618_0001_1][Event:DAG_STARTED]: dagID=dag_1463524597618_0001_1, startTime=1463524597776
2016-05-17 22:36:37,777 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:37,777 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:37,777 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524597618_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:37,777 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/0_'field'_ALL, applicationId=application_1463524597618_0001, dagName=[D1511BC5B7F34357B4556137389C072C/D75AABD94B3E4B21B818DC3DA7957B83] each/(1/1) ...blyplatform/0_'field'_ALL
2016-05-17 22:36:37,777 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D3B4F6252A7747918D2D64D018FAC446 : {4DF6962B14A14A909B801F20493B8091={InputName=4DF6962B14A14A909B801F20493B8091}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:37,777 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 4DF6962B14A14A909B801F20493B8091, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:37,777 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446]
2016-05-17 22:36:37,778 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446]
2016-05-17 22:36:37,778 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446]
2016-05-17 22:36:37,778 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446]. Starting root input initializers: 1
2016-05-17 22:36:37,778 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:37,779 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 4DF6962B14A14A909B801F20493B8091 on vertex vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446]
2016-05-17 22:36:37,781 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:37,782 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 4DF6962B14A14A909B801F20493B8091 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:37,782 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:37,782 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:37,784 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:37,784 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 4DF6962B14A14A909B801F20493B8091 on vertex vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446]
2016-05-17 22:36:37,785 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:37,785 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {4DF6962B14A14A909B801F20493B8091=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:37,785 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446] parallelism set to 1
2016-05-17 22:36:37,785 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:37,785 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597618_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524597618_0001_1_00, eventCount=1
2016-05-17 22:36:37,785 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446]
2016-05-17 22:36:37,786 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=11ADDC370C1E400FA064576CB56E9F1F, vertexId=vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:37,786 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=11ADDC370C1E400FA064576CB56E9F1F, vertexId=vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446]
2016-05-17 22:36:37,788 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=11ADDC370C1E400FA064576CB56E9F1F, vertexName=D3B4F6252A7747918D2D64D018FAC446, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:37,789 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=11ADDC370C1E400FA064576CB56E9F1F, vertexId=vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446]
2016-05-17 22:36:37,790 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:37,790 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597618_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D3B4F6252A7747918D2D64D018FAC446, vertexId=vertex_1463524597618_0001_1_00, initRequestedTime=1463524597777, initedTime=1463524597790, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:37,790 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446] with distanceFromRoot: 0
2016-05-17 22:36:37,790 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:37,791 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D3B4F6252A7747918D2D64D018FAC446
2016-05-17 22:36:37,791 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446]
2016-05-17 22:36:37,791 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597618_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524597618_0001_1_00, startRequestedTime=1463524597791, startedTime=1463524597791
2016-05-17 22:36:37,791 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:37,792 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:37,792 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:37,793 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597618_0001_1][Event:TASK_STARTED]: vertexName=D3B4F6252A7747918D2D64D018FAC446, taskId=task_1463524597618_0001_1_00_000000, scheduledTime=1463524597792, launchTime=1463524597792
2016-05-17 22:36:37,793 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524597618_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:37,793 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446]
2016-05-17 22:36:37,793 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D1511BC5B7F34357B4556137389C072C/D75AABD94B3E4B21B818DC3DA7957B83] each/(1/1) ...blyplatform/0_'field'_ALL, VertexName: D3B4F6252A7747918D2D64D018FAC446, VertexParallelism: 1, TaskAttemptID:attempt_1463524597618_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4DF6962B14A14A909B801F20493B8091, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=11ADDC370C1E400FA064576CB56E9F1F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:37,794 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524597618_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:37,794 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:37,794 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:37,795 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:37,795 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:37,796 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524597618_0001_00_000001
2016-05-17 22:36:37,796 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524597618_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:37,796 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524597618_0001, with dag id: dag_1463524597618_0001_1
2016-05-17 22:36:37,797 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524597618_0001_00_000001
2016-05-17 22:36:37,797 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:37,798 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524597618_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:37,799 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:37,799 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597618_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524597618_0001_00_000001, launchTime=1463524597798
2016-05-17 22:36:37,801 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524597618_0001_00_000001
2016-05-17 22:36:37,801 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524597618_0001_1_00_000000_0] to container: [container_1463524597618_0001_00_000001]
2016-05-17 22:36:37,801 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524597618_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:37,802 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524597618_0001_00_000001 given task: attempt_1463524597618_0001_1_00_000000_0
2016-05-17 22:36:37,802 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524597618_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524597618_0001_1_00_000000_0
2016-05-17 22:36:37,802 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524597618_0001_1_00_000000_0] started. Is using containerId: [container_1463524597618_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:37,802 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597618_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D3B4F6252A7747918D2D64D018FAC446, taskAttemptId=attempt_1463524597618_0001_1_00_000000_0, startTime=1463524597802, containerId=container_1463524597618_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524597618_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:37,802 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:37,802 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524597618_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:37,803 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524597618_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:37 Starting to run new task attempt: attempt_1463524597618_0001_1_00_000000_0
2016-05-17 22:36:37,803 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524597618_0001_1_00_000000_0
2016-05-17 22:36:37,803 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:37,804 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:37,804 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:37,804 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:37,804 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:37,804 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:37,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D1511BC5B7F34357B4556137389C072C/D75AABD94B3E4B21B818DC3DA7957B83] each/(1/1) ...blyplatform/0_'field'_ALL, VertexName: D3B4F6252A7747918D2D64D018FAC446, VertexParallelism: 1, TaskAttemptID:attempt_1463524597618_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4DF6962B14A14A909B801F20493B8091, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=11ADDC370C1E400FA064576CB56E9F1F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:37,805 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:37,805 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524597618_0001_1_00_000000_0
2016-05-17 22:36:37,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:37,806 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:37,806 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524597618_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:37,806 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:37,807 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=11ADDC370C1E400FA064576CB56E9F1F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:37,807 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:37,808 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 11ADDC370C1E400FA064576CB56E9F1F
2016-05-17 22:36:37,808 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:37,808 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 11ADDC370C1E400FA064576CB56E9F1F
2016-05-17 22:36:37,809 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=4DF6962B14A14A909B801F20493B8091, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:37,809 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:37,809 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 4DF6962B14A14A909B801F20493B8091
2016-05-17 22:36:37,809 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:37,810 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 4DF6962B14A14A909B801F20493B8091
2016-05-17 22:36:37,812 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:37,813 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 4DF6962B14A14A909B801F20493B8091
2016-05-17 22:36:37,813 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 4DF6962B14A14A909B801F20493B8091
2016-05-17 22:36:37,818 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D3B4F6252A7747918D2D64D018FAC446, ordinal: 0
2016-05-17 22:36:37,819 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D3B4F6252A7747918D2D64D018FAC446, mem on start (mb), free: 130, total: 389, max: 672
2016-05-17 22:36:37,819 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:37,819 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:37,819 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:37,820 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:37,820 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 11ADDC370C1E400FA064576CB56E9F1F
2016-05-17 22:36:37,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:37,821 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:37,821 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:37,821 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:37,822 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:37,822 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:37,822 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:37,822 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:37,823 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 11ADDC370C1E400FA064576CB56E9F1F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:37,823 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 4DF6962B14A14A909B801F20493B8091, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:37,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:37,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 4DF6962B14A14A909B801F20493B8091
2016-05-17 22:36:37,823 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:37,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 4DF6962B14A14A909B801F20493B8091
2016-05-17 22:36:37,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 4DF6962B14A14A909B801F20493B8091 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:37,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:37,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:37,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:37,825 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524597618_0001_1_00_000000_0
2016-05-17 22:36:37,825 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:37,826 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4ddc5c0
2016-05-17 22:36:37,829 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:37,829 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4ddc5c0
2016-05-17 22:36:37,829 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:37,829 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:37,829 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 4DF6962B14A14A909B801F20493B8091
2016-05-17 22:36:37,830 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_ALL"], id: 11ADDC370C1E400FA064576CB56E9F1F
2016-05-17 22:36:37,830 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_ALL"]
2016-05-17 22:36:37,830 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:37,830 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:37,831 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D3B4F6252A7747918D2D64D018FAC446, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:37,834 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:37,834 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:37,835 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524597618_0001_1_00_000000_0
2016-05-17 22:36:37,835 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524597618_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:37,836 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524597618_0001_1_00_000000_0
2016-05-17 22:36:37,836 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D3B4F6252A7747918D2D64D018FAC446, mem on close (mb), free: 129, total: 389, max: 672
2016-05-17 22:36:37 Completed running task attempt: attempt_1463524597618_0001_1_00_000000_0
2016-05-17 22:36:37,837 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524597618_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:37,838 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597618_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D3B4F6252A7747918D2D64D018FAC446, taskAttemptId=attempt_1463524597618_0001_1_00_000000_0, startTime=1463524597802, finishTime=1463524597838, timeTaken=36, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407896064, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524597831, Process_Duration=5, Process_End_Time=1463524597836, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:37,839 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524597618_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:37,839 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:37,839 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524597618_0001_1_00_000000_0
2016-05-17 22:36:37,850 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597618_0001_1][Event:TASK_FINISHED]: vertexName=D3B4F6252A7747918D2D64D018FAC446, taskId=task_1463524597618_0001_1_00_000000, startTime=1463524597802, finishTime=1463524597850, timeTaken=48, status=SUCCEEDED, successfulAttemptID=attempt_1463524597618_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407896064, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524597831, Process_Duration=5, Process_End_Time=1463524597836, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:37,850 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:37,851 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524597618_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:37,839 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:37,853 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446] : 1
2016-05-17 22:36:37,853 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446]
2016-05-17 22:36:37,856 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407896064, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524597831, Process_Duration=5, Process_End_Time=1463524597836, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:37,856 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:37,856 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:37,856 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:37,856 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597618_0001_1][Event:VERTEX_FINISHED]: vertexName=D3B4F6252A7747918D2D64D018FAC446, vertexId=vertex_1463524597618_0001_1_00, initRequestedTime=1463524597777, initedTime=1463524597790, startRequestedTime=1463524597791, startedTime=1463524597791, finishTime=1463524597853, timeTaken=62, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407896064, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524597831, Process_Duration=5, Process_End_Time=1463524597836, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524597802, firstTasksToStart=[ task_1463524597618_0001_1_00_000000 ], lastTaskFinishTime=1463524597838, lastTasksToFinish=[ task_1463524597618_0001_1_00_000000 ], minTaskDuration=36, maxTaskDuration=36, avgTaskDuration=36.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524597618_0001_1_00_000000 ], longestDurationTasks=[ task_1463524597618_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:37,857 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524597618_0001_00_000001
2016-05-17 22:36:37,857 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:37,857 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524597618_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:37,857 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597618_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524597618_0001_00_000001, stoppedTime=1463524597857, exitStatus=0
2016-05-17 22:36:37,858 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524597618_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:37,858 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:37,858 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524597618_0001_1
2016-05-17 22:36:37,858 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597618_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524597618_0001_1
2016-05-17 22:36:37,858 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:37,858 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 11ADDC370C1E400FA064576CB56E9F1F for vertex: vertex_1463524597618_0001_1_00 [D3B4F6252A7747918D2D64D018FAC446]
2016-05-17 22:36:37,867 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524597618_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:37,867 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524597618_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:37,867 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524597618_0001_00_000001, Exiting
2016-05-17 22:36:37,867 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524597618_0001_00_000001
2016-05-17 22:36:37,867 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524597618_0001_00_000001
2016-05-17 22:36:37,868 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524597618_0001_1, queueSize=0
2016-05-17 22:36:37,868 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:37,868 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524597618_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524597618_0001_1, startTime=1463524597776, finishTime=1463524597866, timeTaken=90, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407896064, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524597831, Process_Duration=5, Process_End_Time=1463524597836, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:37,868 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524597618_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:37,869 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524597618_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:37,869 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524597618_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:36:37 Completed Dag: dag_1463524597618_0001_1
2016-05-17 22:36:37,869 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524597618_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:37,869 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524597618_0001_1_post
2016-05-17 22:36:37,869 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:37,869 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:37,869 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524597618_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:36:37,870 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524597618_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:37,870 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524597618_0001_00_000001 completed successfully
2016-05-17 22:36:37,870 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524597618_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:36:37,876 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/0_'field'_ALL, applicationId=application_1463524597618_0001
2016-05-17 22:36:37,878 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:37,878 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:37,878 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:37,878 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:37,878 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:37,879 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:37,879 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:37,879 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:37,880 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:37,880 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:37,880 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:37,881 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:37,881 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:37,885 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D1511BC5B7F34357B4556137389C072C/D75AABD94B3E4B21B818DC3DA7957B83/.tez/application_1463524597618_0001
2016-05-17 22:36:37,885 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_ALL/_temporary
2016-05-17 22:36:37,886 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.292
2016-05-17 22:36:37,950 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_'field'_ALL
2016-05-17 22:36:37,970 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:37,970 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:37,986 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:37,986 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:37,998 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-05-17 22:36:37,999 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-05-17 22:36:38,000 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:38,000 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:38,000 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_'field'_REPLACE
2016-05-17 22:36:38,020 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:38,020 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:38,054 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 22:36:38,059 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-17 22:36:38,060 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:38,060 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:38,061 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:38,080 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:38,081 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_SWAP
2016-05-17 22:36:38,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:38,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:38,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_SWAP"]
2016-05-17 22:36:38,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:38,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:38,096 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:38,097 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...lyplatform/0_'field'_SWAP
2016-05-17 22:36:38,097 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:38,114 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:38,120 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:38,121 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:38,121 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:38,132 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5BE0456E6E8B44BD828EF296336585F5/B826E9C4568E4EAB93623B0BA13D44A7/.tez/application_1463524598121_0001 doesn't exist and is created
2016-05-17 22:36:38,156 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:38,156 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:38,157 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5BE0456E6E8B44BD828EF296336585F5/B826E9C4568E4EAB93623B0BA13D44A7/.tez/application_1463524598121_0001
2016-05-17 22:36:38,161 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524598121_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:38,178 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:38,179 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:38,181 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:38,181 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:38,181 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5BE0456E6E8B44BD828EF296336585F5/B826E9C4568E4EAB93623B0BA13D44A7/.tez/application_1463524598121_0001/localmode-log-dir
2016-05-17 22:36:38,182 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5BE0456E6E8B44BD828EF296336585F5/B826E9C4568E4EAB93623B0BA13D44A7/.tez/application_1463524598121_0001/localmode-log-dir/history.txt.appattempt_1463524598121_0001_000000, maxErrors=10
2016-05-17 22:36:38,182 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:38,182 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524598121_0001_000000, appSubmitTime=1463524598160, launchTime=1463524598160
2016-05-17 22:36:38,186 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:38,188 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:38,189 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:38,189 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:38,193 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:56639
2016-05-17 22:36:38,200 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:38,202 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524598121_0001_000000, startTime=1463524598202
2016-05-17 22:36:38,202 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:38,202 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:38,257 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:38,257 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:38,257 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/0_'field'_SWAP, applicationId=application_1463524598121_0001, dagName=[5BE0456E6E8B44BD828EF296336585F5/B826E9C4568E4EAB93623B0BA13D44A7] each/(1/1) ...lyplatform/0_'field'_SWAP
2016-05-17 22:36:38,272 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [5BE0456E6E8B44BD828EF296336585F5/B826E9C4568E4EAB93623B0BA13D44A7] each/(1/1) ...lyplatform/0_'field'_SWAP
2016-05-17 22:36:38,274 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524598121_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5BE0456E6E8B44BD828EF296336585F5/B826E9C4568E4EAB93623B0BA13D44A7/.tez/application_1463524598121_0001/localmode-log-dir/dag_1463524598121_0001_1.dot
2016-05-17 22:36:38,274 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5BE0456E6E8B44BD828EF296336585F5/B826E9C4568E4EAB93623B0BA13D44A7/.tez/application_1463524598121_0001/localmode-log-dir/dag_1463524598121_0001_1-tez-dag.pb.txt
2016-05-17 22:36:38,278 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524598121_0001_1
2016-05-17 22:36:38,278 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:38,278 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [5BE0456E6E8B44BD828EF296336585F5/B826E9C4568E4EAB93623B0BA13D44A7] each/(1/1) ...lyplatform/0_'field'_SWAP
2016-05-17 22:36:38 Running Dag: dag_1463524598121_0001_1
2016-05-17 22:36:38,292 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598121_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524598121_0001_1, submitTime=1463524598272
2016-05-17 22:36:38,292 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:38,292 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FB9E32661CA742CEBF4B3AC72F4CD3A8
2016-05-17 22:36:38,293 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:38,294 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598121_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524598121_0001_1, initTime=1463524598292
2016-05-17 22:36:38,294 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:38,294 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524598121_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:38,294 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598121_0001_1][Event:DAG_STARTED]: dagID=dag_1463524598121_0001_1, startTime=1463524598294
2016-05-17 22:36:38,294 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:38,294 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:38,294 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524598121_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:38,295 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FB9E32661CA742CEBF4B3AC72F4CD3A8 : {DCF1649EC9F846EF9F12832B564262F3={InputName=DCF1649EC9F846EF9F12832B564262F3}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:38,295 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: DCF1649EC9F846EF9F12832B564262F3, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:38,295 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8]
2016-05-17 22:36:38,295 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8]
2016-05-17 22:36:38,295 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8]
2016-05-17 22:36:38,295 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8]. Starting root input initializers: 1
2016-05-17 22:36:38,296 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:38,296 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: DCF1649EC9F846EF9F12832B564262F3 on vertex vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8]
2016-05-17 22:36:38,297 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/0_'field'_SWAP, applicationId=application_1463524598121_0001, dagName=[5BE0456E6E8B44BD828EF296336585F5/B826E9C4568E4EAB93623B0BA13D44A7] each/(1/1) ...lyplatform/0_'field'_SWAP
2016-05-17 22:36:38,298 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:38,298 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input DCF1649EC9F846EF9F12832B564262F3 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:38,299 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:38,299 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:38,300 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:38,300 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: DCF1649EC9F846EF9F12832B564262F3 on vertex vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8]
2016-05-17 22:36:38,301 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:38,301 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {DCF1649EC9F846EF9F12832B564262F3=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:38,301 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8] parallelism set to 1
2016-05-17 22:36:38,301 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:38,301 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598121_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524598121_0001_1_00, eventCount=1
2016-05-17 22:36:38,302 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8]
2016-05-17 22:36:38,302 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=B85D6EEF2CE94B039188B5945AB037F3, vertexId=vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:38,302 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=B85D6EEF2CE94B039188B5945AB037F3, vertexId=vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8]
2016-05-17 22:36:38,304 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=B85D6EEF2CE94B039188B5945AB037F3, vertexName=FB9E32661CA742CEBF4B3AC72F4CD3A8, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:38,304 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=B85D6EEF2CE94B039188B5945AB037F3, vertexId=vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8]
2016-05-17 22:36:38,306 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598121_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FB9E32661CA742CEBF4B3AC72F4CD3A8, vertexId=vertex_1463524598121_0001_1_00, initRequestedTime=1463524598295, initedTime=1463524598305, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:38,306 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8] with distanceFromRoot: 0
2016-05-17 22:36:38,306 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:38,306 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:38,306 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FB9E32661CA742CEBF4B3AC72F4CD3A8
2016-05-17 22:36:38,306 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8]
2016-05-17 22:36:38,307 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598121_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524598121_0001_1_00, startRequestedTime=1463524598306, startedTime=1463524598306
2016-05-17 22:36:38,307 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:38,307 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:38,307 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:38,307 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598121_0001_1][Event:TASK_STARTED]: vertexName=FB9E32661CA742CEBF4B3AC72F4CD3A8, taskId=task_1463524598121_0001_1_00_000000, scheduledTime=1463524598307, launchTime=1463524598307
2016-05-17 22:36:38,308 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524598121_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:38,308 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8]
2016-05-17 22:36:38,308 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5BE0456E6E8B44BD828EF296336585F5/B826E9C4568E4EAB93623B0BA13D44A7] each/(1/1) ...lyplatform/0_'field'_SWAP, VertexName: FB9E32661CA742CEBF4B3AC72F4CD3A8, VertexParallelism: 1, TaskAttemptID:attempt_1463524598121_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DCF1649EC9F846EF9F12832B564262F3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B85D6EEF2CE94B039188B5945AB037F3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:38,309 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524598121_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:38,309 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:38,310 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:38,310 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:38,310 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:38,311 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524598121_0001_00_000001
2016-05-17 22:36:38,311 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524598121_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:38,312 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524598121_0001_00_000001
2016-05-17 22:36:38,312 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:38,313 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524598121_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:38,313 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598121_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524598121_0001_00_000001, launchTime=1463524598313
2016-05-17 22:36:38,313 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:38,314 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524598121_0001_00_000001
2016-05-17 22:36:38,315 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524598121_0001_1_00_000000_0] to container: [container_1463524598121_0001_00_000001]
2016-05-17 22:36:38,315 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524598121_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:38,315 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524598121_0001_1_00_000000_0] started. Is using containerId: [container_1463524598121_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:38,315 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598121_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FB9E32661CA742CEBF4B3AC72F4CD3A8, taskAttemptId=attempt_1463524598121_0001_1_00_000000_0, startTime=1463524598315, containerId=container_1463524598121_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524598121_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:38,316 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524598121_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:38,316 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524598121_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:38,316 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:38,316 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524598121_0001_00_000001 given task: attempt_1463524598121_0001_1_00_000000_0
2016-05-17 22:36:38,316 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524598121_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524598121_0001_1_00_000000_0
2016-05-17 22:36:38 Starting to run new task attempt: attempt_1463524598121_0001_1_00_000000_0
2016-05-17 22:36:38,317 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524598121_0001_1_00_000000_0
2016-05-17 22:36:38,317 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:38,317 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:38,317 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:38,317 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:38,317 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:38,318 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:38,318 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5BE0456E6E8B44BD828EF296336585F5/B826E9C4568E4EAB93623B0BA13D44A7] each/(1/1) ...lyplatform/0_'field'_SWAP, VertexName: FB9E32661CA742CEBF4B3AC72F4CD3A8, VertexParallelism: 1, TaskAttemptID:attempt_1463524598121_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DCF1649EC9F846EF9F12832B564262F3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B85D6EEF2CE94B039188B5945AB037F3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:38,318 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:38,320 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524598121_0001, with dag id: dag_1463524598121_0001_1
2016-05-17 22:36:38,325 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524598121_0001_1_00_000000_0
2016-05-17 22:36:38,325 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:38,325 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:38,327 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=DCF1649EC9F846EF9F12832B564262F3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:38,328 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B85D6EEF2CE94B039188B5945AB037F3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:38,329 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:38,329 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: DCF1649EC9F846EF9F12832B564262F3
2016-05-17 22:36:38,329 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: DCF1649EC9F846EF9F12832B564262F3
2016-05-17 22:36:38,331 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:38,331 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: DCF1649EC9F846EF9F12832B564262F3
2016-05-17 22:36:38,331 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: DCF1649EC9F846EF9F12832B564262F3
2016-05-17 22:36:38,329 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:38,328 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:38,327 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524598121_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:38,332 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B85D6EEF2CE94B039188B5945AB037F3
2016-05-17 22:36:38,332 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:38,333 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: B85D6EEF2CE94B039188B5945AB037F3
2016-05-17 22:36:38,335 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:38,343 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:38,344 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B85D6EEF2CE94B039188B5945AB037F3
2016-05-17 22:36:38,344 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FB9E32661CA742CEBF4B3AC72F4CD3A8, ordinal: 0
2016-05-17 22:36:38,345 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FB9E32661CA742CEBF4B3AC72F4CD3A8, mem on start (mb), free: 239, total: 381, max: 672
2016-05-17 22:36:38,345 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:38,345 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:38,346 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:38,346 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:38,346 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:38,346 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:38,347 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:38,347 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:38,347 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:38,347 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:38,348 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:38,348 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, DCF1649EC9F846EF9F12832B564262F3, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:38,348 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B85D6EEF2CE94B039188B5945AB037F3, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:38,348 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:38,348 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: DCF1649EC9F846EF9F12832B564262F3
2016-05-17 22:36:38,348 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:38,348 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: DCF1649EC9F846EF9F12832B564262F3
2016-05-17 22:36:38,349 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: DCF1649EC9F846EF9F12832B564262F3 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:38,349 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:38,349 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:38,349 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:38,350 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524598121_0001_1_00_000000_0
2016-05-17 22:36:38,350 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:38,351 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7ecf2971
2016-05-17 22:36:38,351 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7ecf2971
2016-05-17 22:36:38,352 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:38,352 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:38,354 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:38,355 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: DCF1649EC9F846EF9F12832B564262F3
2016-05-17 22:36:38,355 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_SWAP"], id: B85D6EEF2CE94B039188B5945AB037F3
2016-05-17 22:36:38,355 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_SWAP"]
2016-05-17 22:36:38,356 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:38,356 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:38,356 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FB9E32661CA742CEBF4B3AC72F4CD3A8, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:38,359 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:38,359 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:38,359 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524598121_0001_1_00_000000_0
2016-05-17 22:36:38,359 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524598121_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:38,360 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524598121_0001_1_00_000000_0
2016-05-17 22:36:38,360 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FB9E32661CA742CEBF4B3AC72F4CD3A8, mem on close (mb), free: 238, total: 381, max: 672
2016-05-17 22:36:38 Completed running task attempt: attempt_1463524598121_0001_1_00_000000_0
2016-05-17 22:36:38,361 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524598121_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:38,363 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598121_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FB9E32661CA742CEBF4B3AC72F4CD3A8, taskAttemptId=attempt_1463524598121_0001_1_00_000000_0, startTime=1463524598315, finishTime=1463524598362, timeTaken=47, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=399507456, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524598356, Process_Duration=4, Process_End_Time=1463524598360, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:38,363 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524598121_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:38,363 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524598121_0001_1_00_000000_0
2016-05-17 22:36:38,363 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:38,364 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598121_0001_1][Event:TASK_FINISHED]: vertexName=FB9E32661CA742CEBF4B3AC72F4CD3A8, taskId=task_1463524598121_0001_1_00_000000, startTime=1463524598315, finishTime=1463524598363, timeTaken=48, status=SUCCEEDED, successfulAttemptID=attempt_1463524598121_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=399507456, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524598356, Process_Duration=4, Process_End_Time=1463524598360, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:38,364 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524598121_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:38,364 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8] : 1
2016-05-17 22:36:38,364 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8]
2016-05-17 22:36:38,365 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:38,365 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598121_0001_1][Event:VERTEX_FINISHED]: vertexName=FB9E32661CA742CEBF4B3AC72F4CD3A8, vertexId=vertex_1463524598121_0001_1_00, initRequestedTime=1463524598295, initedTime=1463524598305, startRequestedTime=1463524598306, startedTime=1463524598306, finishTime=1463524598364, timeTaken=58, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=399507456, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524598356, Process_Duration=4, Process_End_Time=1463524598360, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524598315, firstTasksToStart=[ task_1463524598121_0001_1_00_000000 ], lastTaskFinishTime=1463524598362, lastTasksToFinish=[ task_1463524598121_0001_1_00_000000 ], minTaskDuration=47, maxTaskDuration=47, avgTaskDuration=47.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524598121_0001_1_00_000000 ], longestDurationTasks=[ task_1463524598121_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:38,365 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:38,365 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:38,365 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524598121_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:38,366 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:38,366 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598121_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524598121_0001_00_000001, stoppedTime=1463524598366, exitStatus=0
2016-05-17 22:36:38,366 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524598121_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:38,366 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:38,366 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524598121_0001_1
2016-05-17 22:36:38,366 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598121_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524598121_0001_1
2016-05-17 22:36:38,366 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: B85D6EEF2CE94B039188B5945AB037F3 for vertex: vertex_1463524598121_0001_1_00 [FB9E32661CA742CEBF4B3AC72F4CD3A8]
2016-05-17 22:36:38,366 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:38,368 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=399507456, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524598356, Process_Duration=4, Process_End_Time=1463524598360, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:38,370 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:38,370 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:38,370 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524598121_0001_00_000001
2016-05-17 22:36:38,371 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524598121_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:38,371 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524598121_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:38,371 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524598121_0001_00_000001, Exiting
2016-05-17 22:36:38,371 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524598121_0001_00_000001
2016-05-17 22:36:38,371 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524598121_0001_00_000001
2016-05-17 22:36:38,372 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524598121_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:38,372 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524598121_0001_00_000001 completed successfully
2016-05-17 22:36:38,375 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524598121_0001_1, queueSize=0
2016-05-17 22:36:38,376 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598121_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524598121_0001_1, startTime=1463524598294, finishTime=1463524598375, timeTaken=81, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=399507456, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524598356, Process_Duration=4, Process_End_Time=1463524598360, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:38,376 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:38,376 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524598121_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:38,376 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524598121_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:38,376 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524598121_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:38 Completed Dag: dag_1463524598121_0001_1
2016-05-17 22:36:38,376 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524598121_0001_00_000001
2016-05-17 22:36:38,377 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524598121_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:38,377 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524598121_0001_1_post
2016-05-17 22:36:38,377 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:38,377 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:38,382 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/0_'field'_SWAP, applicationId=application_1463524598121_0001
2016-05-17 22:36:38,384 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:38,384 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:38,384 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:38,384 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:38,384 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:38,384 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:38,385 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:38,385 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:38,385 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:38,385 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:38,385 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:38,386 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:38,386 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:38,389 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5BE0456E6E8B44BD828EF296336585F5/B826E9C4568E4EAB93623B0BA13D44A7/.tez/application_1463524598121_0001
2016-05-17 22:36:38,390 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_'field'_SWAP/_temporary
2016-05-17 22:36:38,390 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.296
2016-05-17 22:36:38,408 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:38,408 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:38,453 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_'field'_SWAP
2016-05-17 22:36:38,473 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:38,473 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:38,507 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 22:36:38,512 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-17 22:36:38,512 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:38,513 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:38,513 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:38,525 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:38,525 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_-1
2016-05-17 22:36:38,534 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:38,534 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:38,534 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_-1"]
2016-05-17 22:36:38,535 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:38,535 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:38,535 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:38,536 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...mblyplatform/0_UNKNOWN_-1
2016-05-17 22:36:38,537 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:38,548 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:38,552 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:38,553 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:38,553 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:38,562 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/77E6850526E0439981E9AFBC5F72B38B/B402153F399A479E8E5AF21340B89CC7/.tez/application_1463524598553_0001 doesn't exist and is created
2016-05-17 22:36:38,577 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:38,577 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:38,577 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/77E6850526E0439981E9AFBC5F72B38B/B402153F399A479E8E5AF21340B89CC7/.tez/application_1463524598553_0001
2016-05-17 22:36:38,580 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524598553_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:38,591 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:38,592 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:38,593 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:38,593 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:38,598 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/77E6850526E0439981E9AFBC5F72B38B/B402153F399A479E8E5AF21340B89CC7/.tez/application_1463524598553_0001/localmode-log-dir
2016-05-17 22:36:38,598 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/77E6850526E0439981E9AFBC5F72B38B/B402153F399A479E8E5AF21340B89CC7/.tez/application_1463524598553_0001/localmode-log-dir/history.txt.appattempt_1463524598553_0001_000000, maxErrors=10
2016-05-17 22:36:38,598 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:38,598 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524598553_0001_000000, appSubmitTime=1463524598580, launchTime=1463524598580
2016-05-17 22:36:38,602 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:38,602 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:38,604 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:38,604 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:38,605 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:45614
2016-05-17 22:36:38,608 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:38,609 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524598553_0001_000000, startTime=1463524598609
2016-05-17 22:36:38,609 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:38,609 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:38,677 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:38,677 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:38,678 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...mblyplatform/0_UNKNOWN_-1, applicationId=application_1463524598553_0001, dagName=[77E6850526E0439981E9AFBC5F72B38B/B402153F399A479E8E5AF21340B89CC7] each/(1/1) ...mblyplatform/0_UNKNOWN_-1
2016-05-17 22:36:38,693 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [77E6850526E0439981E9AFBC5F72B38B/B402153F399A479E8E5AF21340B89CC7] each/(1/1) ...mblyplatform/0_UNKNOWN_-1
2016-05-17 22:36:38,694 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524598553_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/77E6850526E0439981E9AFBC5F72B38B/B402153F399A479E8E5AF21340B89CC7/.tez/application_1463524598553_0001/localmode-log-dir/dag_1463524598553_0001_1.dot
2016-05-17 22:36:38,694 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/77E6850526E0439981E9AFBC5F72B38B/B402153F399A479E8E5AF21340B89CC7/.tez/application_1463524598553_0001/localmode-log-dir/dag_1463524598553_0001_1-tez-dag.pb.txt
2016-05-17 22:36:38,699 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524598553_0001_1
2016-05-17 22:36:38,699 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:38,699 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [77E6850526E0439981E9AFBC5F72B38B/B402153F399A479E8E5AF21340B89CC7] each/(1/1) ...mblyplatform/0_UNKNOWN_-1
2016-05-17 22:36:38 Running Dag: dag_1463524598553_0001_1
2016-05-17 22:36:38,711 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:38,712 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598553_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524598553_0001_1, submitTime=1463524598693
2016-05-17 22:36:38,712 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A6CA53E199DF45028E909898A8559882
2016-05-17 22:36:38,712 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:38,712 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:38,713 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598553_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524598553_0001_1, initTime=1463524598712
2016-05-17 22:36:38,713 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524598553_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:38,713 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598553_0001_1][Event:DAG_STARTED]: dagID=dag_1463524598553_0001_1, startTime=1463524598713
2016-05-17 22:36:38,713 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:38,713 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...mblyplatform/0_UNKNOWN_-1, applicationId=application_1463524598553_0001, dagName=[77E6850526E0439981E9AFBC5F72B38B/B402153F399A479E8E5AF21340B89CC7] each/(1/1) ...mblyplatform/0_UNKNOWN_-1
2016-05-17 22:36:38,713 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:38,713 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524598553_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:38,714 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A6CA53E199DF45028E909898A8559882 : {C60C46E7BDA24E1BB608F53E30D96445={InputName=C60C46E7BDA24E1BB608F53E30D96445}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:38,714 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: C60C46E7BDA24E1BB608F53E30D96445, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:38,714 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882]
2016-05-17 22:36:38,714 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882]
2016-05-17 22:36:38,714 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882]
2016-05-17 22:36:38,714 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882]. Starting root input initializers: 1
2016-05-17 22:36:38,715 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:38,715 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: C60C46E7BDA24E1BB608F53E30D96445 on vertex vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882]
2016-05-17 22:36:38,716 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:38,717 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input C60C46E7BDA24E1BB608F53E30D96445 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:38,717 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:38,718 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:38,719 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:38,719 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: C60C46E7BDA24E1BB608F53E30D96445 on vertex vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882]
2016-05-17 22:36:38,719 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:38,719 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {C60C46E7BDA24E1BB608F53E30D96445=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:38,720 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882] parallelism set to 1
2016-05-17 22:36:38,720 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:38,720 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598553_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524598553_0001_1_00, eventCount=1
2016-05-17 22:36:38,720 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882]
2016-05-17 22:36:38,720 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=63E421A1B3CC41019C191F6DA830EC63, vertexId=vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:38,720 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=63E421A1B3CC41019C191F6DA830EC63, vertexId=vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882]
2016-05-17 22:36:38,722 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=63E421A1B3CC41019C191F6DA830EC63, vertexName=A6CA53E199DF45028E909898A8559882, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:38,722 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=63E421A1B3CC41019C191F6DA830EC63, vertexId=vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882]
2016-05-17 22:36:38,723 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:38,723 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598553_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A6CA53E199DF45028E909898A8559882, vertexId=vertex_1463524598553_0001_1_00, initRequestedTime=1463524598714, initedTime=1463524598723, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:38,723 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882] with distanceFromRoot: 0
2016-05-17 22:36:38,723 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:38,724 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A6CA53E199DF45028E909898A8559882
2016-05-17 22:36:38,724 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882]
2016-05-17 22:36:38,724 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:38,724 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598553_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524598553_0001_1_00, startRequestedTime=1463524598724, startedTime=1463524598724
2016-05-17 22:36:38,725 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:38,725 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:38,725 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598553_0001_1][Event:TASK_STARTED]: vertexName=A6CA53E199DF45028E909898A8559882, taskId=task_1463524598553_0001_1_00_000000, scheduledTime=1463524598725, launchTime=1463524598725
2016-05-17 22:36:38,726 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524598553_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:38,726 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882]
2016-05-17 22:36:38,726 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [77E6850526E0439981E9AFBC5F72B38B/B402153F399A479E8E5AF21340B89CC7] each/(1/1) ...mblyplatform/0_UNKNOWN_-1, VertexName: A6CA53E199DF45028E909898A8559882, VertexParallelism: 1, TaskAttemptID:attempt_1463524598553_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C60C46E7BDA24E1BB608F53E30D96445, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=63E421A1B3CC41019C191F6DA830EC63, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:38,726 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524598553_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:38,727 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:38,727 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:38,728 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:38,728 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:38,728 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524598553_0001_00_000001
2016-05-17 22:36:38,728 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524598553_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:38,730 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524598553_0001_00_000001
2016-05-17 22:36:38,730 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:38,732 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524598553_0001, with dag id: dag_1463524598553_0001_1
2016-05-17 22:36:38,732 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524598553_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:38,733 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:38,733 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598553_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524598553_0001_00_000001, launchTime=1463524598733
2016-05-17 22:36:38,734 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524598553_0001_00_000001
2016-05-17 22:36:38,735 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524598553_0001_1_00_000000_0] to container: [container_1463524598553_0001_00_000001]
2016-05-17 22:36:38,735 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524598553_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:38,736 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524598553_0001_1_00_000000_0] started. Is using containerId: [container_1463524598553_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:38,736 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598553_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A6CA53E199DF45028E909898A8559882, taskAttemptId=attempt_1463524598553_0001_1_00_000000_0, startTime=1463524598735, containerId=container_1463524598553_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524598553_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:38,736 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524598553_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:38,736 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524598553_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:38,736 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:38,737 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524598553_0001_00_000001 given task: attempt_1463524598553_0001_1_00_000000_0
2016-05-17 22:36:38,737 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524598553_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524598553_0001_1_00_000000_0
2016-05-17 22:36:38 Starting to run new task attempt: attempt_1463524598553_0001_1_00_000000_0
2016-05-17 22:36:38,737 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524598553_0001_1_00_000000_0
2016-05-17 22:36:38,737 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:38,738 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:38,738 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:38,738 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:38,738 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:38,738 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:38,738 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [77E6850526E0439981E9AFBC5F72B38B/B402153F399A479E8E5AF21340B89CC7] each/(1/1) ...mblyplatform/0_UNKNOWN_-1, VertexName: A6CA53E199DF45028E909898A8559882, VertexParallelism: 1, TaskAttemptID:attempt_1463524598553_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C60C46E7BDA24E1BB608F53E30D96445, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=63E421A1B3CC41019C191F6DA830EC63, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:38,738 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:38,739 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524598553_0001_1_00_000000_0
2016-05-17 22:36:38,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:38,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:38,740 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524598553_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:38,740 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:38,740 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C60C46E7BDA24E1BB608F53E30D96445, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:38,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:38,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C60C46E7BDA24E1BB608F53E30D96445
2016-05-17 22:36:38,741 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: C60C46E7BDA24E1BB608F53E30D96445
2016-05-17 22:36:38,743 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:38,743 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:38,743 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: C60C46E7BDA24E1BB608F53E30D96445
2016-05-17 22:36:38,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C60C46E7BDA24E1BB608F53E30D96445
2016-05-17 22:36:38,745 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=63E421A1B3CC41019C191F6DA830EC63, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:38,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:38,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 63E421A1B3CC41019C191F6DA830EC63
2016-05-17 22:36:38,746 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:38,747 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 63E421A1B3CC41019C191F6DA830EC63
2016-05-17 22:36:38,756 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A6CA53E199DF45028E909898A8559882, ordinal: 0
2016-05-17 22:36:38,757 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A6CA53E199DF45028E909898A8559882, mem on start (mb), free: 165, total: 381, max: 672
2016-05-17 22:36:38,757 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:38,758 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:38,758 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:38,764 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:38,765 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 63E421A1B3CC41019C191F6DA830EC63
2016-05-17 22:36:38,765 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:38,765 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:38,766 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:38,766 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:38,766 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:38,767 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:38,767 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:38,767 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:38,768 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C60C46E7BDA24E1BB608F53E30D96445, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:38,768 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 63E421A1B3CC41019C191F6DA830EC63, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:38,769 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:38,769 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C60C46E7BDA24E1BB608F53E30D96445
2016-05-17 22:36:38,769 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:38,769 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C60C46E7BDA24E1BB608F53E30D96445
2016-05-17 22:36:38,770 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C60C46E7BDA24E1BB608F53E30D96445 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:38,770 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:38,771 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:38,771 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:38,772 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524598553_0001_1_00_000000_0
2016-05-17 22:36:38,772 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:38,773 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7e2c2e60
2016-05-17 22:36:38,774 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7e2c2e60
2016-05-17 22:36:38,774 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:38,774 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:38,777 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:38,777 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: C60C46E7BDA24E1BB608F53E30D96445
2016-05-17 22:36:38,778 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_-1"], id: 63E421A1B3CC41019C191F6DA830EC63
2016-05-17 22:36:38,778 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_-1"]
2016-05-17 22:36:38,778 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:38,778 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:38,779 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A6CA53E199DF45028E909898A8559882, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:38,782 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:38,782 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:38,782 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524598553_0001_1_00_000000_0
2016-05-17 22:36:38,783 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524598553_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:38,784 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524598553_0001_1_00_000000_0
2016-05-17 22:36:38,784 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A6CA53E199DF45028E909898A8559882, mem on close (mb), free: 164, total: 381, max: 672
2016-05-17 22:36:38 Completed running task attempt: attempt_1463524598553_0001_1_00_000000_0
2016-05-17 22:36:38,785 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524598553_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:38,786 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=399507456, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524598779, Process_Duration=4, Process_End_Time=1463524598783, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:38,786 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:38,786 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598553_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A6CA53E199DF45028E909898A8559882, taskAttemptId=attempt_1463524598553_0001_1_00_000000_0, startTime=1463524598735, finishTime=1463524598786, timeTaken=51, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=399507456, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524598779, Process_Duration=4, Process_End_Time=1463524598783, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:38,787 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524598553_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:38,787 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:38,787 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524598553_0001_1_00_000000_0
2016-05-17 22:36:38,788 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:38,788 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:38,788 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598553_0001_1][Event:TASK_FINISHED]: vertexName=A6CA53E199DF45028E909898A8559882, taskId=task_1463524598553_0001_1_00_000000, startTime=1463524598735, finishTime=1463524598788, timeTaken=53, status=SUCCEEDED, successfulAttemptID=attempt_1463524598553_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=399507456, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524598779, Process_Duration=4, Process_End_Time=1463524598783, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:38,789 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:38,789 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524598553_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:38,789 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882] : 1
2016-05-17 22:36:38,790 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882]
2016-05-17 22:36:38,790 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:38,790 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598553_0001_1][Event:VERTEX_FINISHED]: vertexName=A6CA53E199DF45028E909898A8559882, vertexId=vertex_1463524598553_0001_1_00, initRequestedTime=1463524598714, initedTime=1463524598723, startRequestedTime=1463524598724, startedTime=1463524598724, finishTime=1463524598790, timeTaken=66, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=399507456, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524598779, Process_Duration=4, Process_End_Time=1463524598783, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524598735, firstTasksToStart=[ task_1463524598553_0001_1_00_000000 ], lastTaskFinishTime=1463524598786, lastTasksToFinish=[ task_1463524598553_0001_1_00_000000 ], minTaskDuration=51, maxTaskDuration=51, avgTaskDuration=51.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524598553_0001_1_00_000000 ], longestDurationTasks=[ task_1463524598553_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:38,791 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:38,791 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524598553_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:38,791 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598553_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524598553_0001_00_000001, stoppedTime=1463524598791, exitStatus=0
2016-05-17 22:36:38,791 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:38,791 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524598553_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:38,791 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:38,791 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524598553_0001_1
2016-05-17 22:36:38,791 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524598553_0001_00_000001
2016-05-17 22:36:38,792 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598553_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524598553_0001_1
2016-05-17 22:36:38,792 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 63E421A1B3CC41019C191F6DA830EC63 for vertex: vertex_1463524598553_0001_1_00 [A6CA53E199DF45028E909898A8559882]
2016-05-17 22:36:38,792 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524598553_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:38,792 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524598553_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:38,793 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524598553_0001_00_000001, Exiting
2016-05-17 22:36:38,793 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524598553_0001_00_000001
2016-05-17 22:36:38,793 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524598553_0001_00_000001
2016-05-17 22:36:38,794 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524598553_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:38,794 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524598553_0001_00_000001 completed successfully
2016-05-17 22:36:38,802 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524598553_0001_1, queueSize=0
2016-05-17 22:36:38,802 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:38,802 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524598553_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524598553_0001_1, startTime=1463524598713, finishTime=1463524598800, timeTaken=87, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=399507456, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524598779, Process_Duration=4, Process_End_Time=1463524598783, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:38,802 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524598553_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:38,803 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524598553_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:38,804 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524598553_0001_00_000001
2016-05-17 22:36:38,804 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524598553_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:38 Completed Dag: dag_1463524598553_0001_1
2016-05-17 22:36:38,805 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524598553_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:38,805 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524598553_0001_1_post
2016-05-17 22:36:38,805 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:38,805 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:38,808 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...mblyplatform/0_UNKNOWN_-1, applicationId=application_1463524598553_0001
2016-05-17 22:36:38,809 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:38,809 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:38,810 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:38,811 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:38,811 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:38,811 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:38,811 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:38,811 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:38,811 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:38,812 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:38,812 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:38,812 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:38,816 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:38,817 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:38,818 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:38,822 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/77E6850526E0439981E9AFBC5F72B38B/B402153F399A479E8E5AF21340B89CC7/.tez/application_1463524598553_0001
2016-05-17 22:36:38,822 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_-1/_temporary
2016-05-17 22:36:38,822 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.288
2016-05-17 22:36:38,886 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_UNKNOWN_-1
2016-05-17 22:36:38,908 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:38,908 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:38,934 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-17 22:36:38,938 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.007
2016-05-17 22:36:38,938 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:38,938 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:38,939 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_UNKNOWN_'field'
2016-05-17 22:36:38,960 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:38,961 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:38,989 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 22:36:39,009 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-05-17 22:36:39,009 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:39,010 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:39,010 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:39,028 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:39,028 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_RESULTS
2016-05-17 22:36:39,036 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:39,037 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:39,037 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_RESULTS"]
2016-05-17 22:36:39,037 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:39,038 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:39,038 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:39,039 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...latform/0_UNKNOWN_RESULTS
2016-05-17 22:36:39,039 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:39,050 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:39,054 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:39,055 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:39,055 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:39,065 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/084AD44E1F564E0BBAACAAD1FD6D71E2/D358190F9F2E470AAC1A89902EBFF0B0/.tez/application_1463524599055_0001 doesn't exist and is created
2016-05-17 22:36:39,082 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:39,082 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:39,083 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/084AD44E1F564E0BBAACAAD1FD6D71E2/D358190F9F2E470AAC1A89902EBFF0B0/.tez/application_1463524599055_0001
2016-05-17 22:36:39,085 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524599055_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:39,095 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:39,095 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:39,096 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:39,097 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:39,097 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/084AD44E1F564E0BBAACAAD1FD6D71E2/D358190F9F2E470AAC1A89902EBFF0B0/.tez/application_1463524599055_0001/localmode-log-dir
2016-05-17 22:36:39,097 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/084AD44E1F564E0BBAACAAD1FD6D71E2/D358190F9F2E470AAC1A89902EBFF0B0/.tez/application_1463524599055_0001/localmode-log-dir/history.txt.appattempt_1463524599055_0001_000000, maxErrors=10
2016-05-17 22:36:39,097 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:39,097 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524599055_0001_000000, appSubmitTime=1463524599085, launchTime=1463524599085
2016-05-17 22:36:39,102 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:39,103 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:40731
2016-05-17 22:36:39,106 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:39,107 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:39,107 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:39,108 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:39,110 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524599055_0001_000000, startTime=1463524599110
2016-05-17 22:36:39,110 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:39,110 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:39,182 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:39,183 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:39,183 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/0_UNKNOWN_RESULTS, applicationId=application_1463524599055_0001, dagName=[084AD44E1F564E0BBAACAAD1FD6D71E2/D358190F9F2E470AAC1A89902EBFF0B0] each/(1/1) ...latform/0_UNKNOWN_RESULTS
2016-05-17 22:36:39,198 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [084AD44E1F564E0BBAACAAD1FD6D71E2/D358190F9F2E470AAC1A89902EBFF0B0] each/(1/1) ...latform/0_UNKNOWN_RESULTS
2016-05-17 22:36:39,200 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524599055_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/084AD44E1F564E0BBAACAAD1FD6D71E2/D358190F9F2E470AAC1A89902EBFF0B0/.tez/application_1463524599055_0001/localmode-log-dir/dag_1463524599055_0001_1.dot
2016-05-17 22:36:39,200 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/084AD44E1F564E0BBAACAAD1FD6D71E2/D358190F9F2E470AAC1A89902EBFF0B0/.tez/application_1463524599055_0001/localmode-log-dir/dag_1463524599055_0001_1-tez-dag.pb.txt
2016-05-17 22:36:39,204 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524599055_0001_1
2016-05-17 22:36:39,204 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:39,204 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [084AD44E1F564E0BBAACAAD1FD6D71E2/D358190F9F2E470AAC1A89902EBFF0B0] each/(1/1) ...latform/0_UNKNOWN_RESULTS
2016-05-17 22:36:39 Running Dag: dag_1463524599055_0001_1
2016-05-17 22:36:39,217 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599055_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524599055_0001_1, submitTime=1463524599198
2016-05-17 22:36:39,217 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:39,218 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D2C2754FDE514F49946A1F0B78F4960F
2016-05-17 22:36:39,218 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:39,219 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599055_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524599055_0001_1, initTime=1463524599217
2016-05-17 22:36:39,219 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524599055_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:39,219 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:39,220 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599055_0001_1][Event:DAG_STARTED]: dagID=dag_1463524599055_0001_1, startTime=1463524599220
2016-05-17 22:36:39,220 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:39,220 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524599055_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:39,220 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/0_UNKNOWN_RESULTS, applicationId=application_1463524599055_0001, dagName=[084AD44E1F564E0BBAACAAD1FD6D71E2/D358190F9F2E470AAC1A89902EBFF0B0] each/(1/1) ...latform/0_UNKNOWN_RESULTS
2016-05-17 22:36:39,220 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:39,220 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D2C2754FDE514F49946A1F0B78F4960F : {6D44DCC180E14374BDC9E17C5219328E={InputName=6D44DCC180E14374BDC9E17C5219328E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:39,221 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 6D44DCC180E14374BDC9E17C5219328E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:39,221 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F]
2016-05-17 22:36:39,221 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F]
2016-05-17 22:36:39,221 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F]
2016-05-17 22:36:39,221 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F]. Starting root input initializers: 1
2016-05-17 22:36:39,222 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:39,222 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 6D44DCC180E14374BDC9E17C5219328E on vertex vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F]
2016-05-17 22:36:39,224 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:39,224 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 6D44DCC180E14374BDC9E17C5219328E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:39,224 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:39,225 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:39,226 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:39,226 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 6D44DCC180E14374BDC9E17C5219328E on vertex vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F]
2016-05-17 22:36:39,226 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:39,227 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {6D44DCC180E14374BDC9E17C5219328E=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:39,227 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F] parallelism set to 1
2016-05-17 22:36:39,227 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:39,227 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599055_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524599055_0001_1_00, eventCount=1
2016-05-17 22:36:39,227 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F]
2016-05-17 22:36:39,227 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=7F929A39C40B4F9881B32B980B26D39B, vertexId=vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:39,228 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=7F929A39C40B4F9881B32B980B26D39B, vertexId=vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F]
2016-05-17 22:36:39,229 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=7F929A39C40B4F9881B32B980B26D39B, vertexName=D2C2754FDE514F49946A1F0B78F4960F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:39,230 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=7F929A39C40B4F9881B32B980B26D39B, vertexId=vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F]
2016-05-17 22:36:39,231 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599055_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D2C2754FDE514F49946A1F0B78F4960F, vertexId=vertex_1463524599055_0001_1_00, initRequestedTime=1463524599220, initedTime=1463524599231, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:39,231 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F] with distanceFromRoot: 0
2016-05-17 22:36:39,231 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:39,231 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:39,231 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D2C2754FDE514F49946A1F0B78F4960F
2016-05-17 22:36:39,232 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F]
2016-05-17 22:36:39,232 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:39,232 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599055_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524599055_0001_1_00, startRequestedTime=1463524599231, startedTime=1463524599231
2016-05-17 22:36:39,232 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:39,232 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:39,233 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599055_0001_1][Event:TASK_STARTED]: vertexName=D2C2754FDE514F49946A1F0B78F4960F, taskId=task_1463524599055_0001_1_00_000000, scheduledTime=1463524599232, launchTime=1463524599232
2016-05-17 22:36:39,233 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524599055_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:39,233 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F]
2016-05-17 22:36:39,233 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [084AD44E1F564E0BBAACAAD1FD6D71E2/D358190F9F2E470AAC1A89902EBFF0B0] each/(1/1) ...latform/0_UNKNOWN_RESULTS, VertexName: D2C2754FDE514F49946A1F0B78F4960F, VertexParallelism: 1, TaskAttemptID:attempt_1463524599055_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6D44DCC180E14374BDC9E17C5219328E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=7F929A39C40B4F9881B32B980B26D39B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:39,233 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524599055_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:39,234 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:39,234 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:39,235 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:39,235 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:39,235 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524599055_0001_00_000001
2016-05-17 22:36:39,235 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524599055_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:39,237 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524599055_0001_00_000001
2016-05-17 22:36:39,237 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:39,238 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524599055_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:39,238 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:39,239 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599055_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524599055_0001_00_000001, launchTime=1463524599238
2016-05-17 22:36:39,239 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524599055_0001_00_000001
2016-05-17 22:36:39,240 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524599055_0001_1_00_000000_0] to container: [container_1463524599055_0001_00_000001]
2016-05-17 22:36:39,240 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524599055_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:39,240 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524599055_0001_1_00_000000_0] started. Is using containerId: [container_1463524599055_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:39,241 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599055_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D2C2754FDE514F49946A1F0B78F4960F, taskAttemptId=attempt_1463524599055_0001_1_00_000000_0, startTime=1463524599240, containerId=container_1463524599055_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524599055_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:39,241 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524599055_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:39,241 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524599055_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:39,241 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:39,241 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524599055_0001_00_000001 given task: attempt_1463524599055_0001_1_00_000000_0
2016-05-17 22:36:39,241 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524599055_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524599055_0001_1_00_000000_0
2016-05-17 22:36:39 Starting to run new task attempt: attempt_1463524599055_0001_1_00_000000_0
2016-05-17 22:36:39,242 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524599055_0001_1_00_000000_0
2016-05-17 22:36:39,242 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:39,242 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:39,242 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:39,242 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:39,242 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:39,242 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:39,243 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [084AD44E1F564E0BBAACAAD1FD6D71E2/D358190F9F2E470AAC1A89902EBFF0B0] each/(1/1) ...latform/0_UNKNOWN_RESULTS, VertexName: D2C2754FDE514F49946A1F0B78F4960F, VertexParallelism: 1, TaskAttemptID:attempt_1463524599055_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6D44DCC180E14374BDC9E17C5219328E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=7F929A39C40B4F9881B32B980B26D39B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:39,243 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:39,243 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524599055_0001_1_00_000000_0
2016-05-17 22:36:39,243 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:39,243 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:39,246 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524599055_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:39,247 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=6D44DCC180E14374BDC9E17C5219328E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:39,247 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=7F929A39C40B4F9881B32B980B26D39B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:39,247 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:39,248 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:39,248 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 7F929A39C40B4F9881B32B980B26D39B
2016-05-17 22:36:39,248 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:39,248 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 7F929A39C40B4F9881B32B980B26D39B
2016-05-17 22:36:39,247 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:39,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 6D44DCC180E14374BDC9E17C5219328E
2016-05-17 22:36:39,249 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 6D44DCC180E14374BDC9E17C5219328E
2016-05-17 22:36:39,252 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:39,253 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:39,253 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 6D44DCC180E14374BDC9E17C5219328E
2016-05-17 22:36:39,253 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 6D44DCC180E14374BDC9E17C5219328E
2016-05-17 22:36:39,254 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524599055_0001, with dag id: dag_1463524599055_0001_1
2016-05-17 22:36:39,261 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:39,261 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 7F929A39C40B4F9881B32B980B26D39B
2016-05-17 22:36:39,261 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D2C2754FDE514F49946A1F0B78F4960F, ordinal: 0
2016-05-17 22:36:39,262 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D2C2754FDE514F49946A1F0B78F4960F, mem on start (mb), free: 296, total: 388, max: 672
2016-05-17 22:36:39,262 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:39,262 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:39,262 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:39,262 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:39,262 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:39,262 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:39,262 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:39,262 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:39,263 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:39,263 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:39,263 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:39,263 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 7F929A39C40B4F9881B32B980B26D39B, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:39,263 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 6D44DCC180E14374BDC9E17C5219328E, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:39,263 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:39,263 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 6D44DCC180E14374BDC9E17C5219328E
2016-05-17 22:36:39,263 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:39,263 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 6D44DCC180E14374BDC9E17C5219328E
2016-05-17 22:36:39,264 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 6D44DCC180E14374BDC9E17C5219328E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:39,264 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:39,264 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:39,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:39,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:39,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:39,265 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524599055_0001_1_00_000000_0
2016-05-17 22:36:39,266 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:39,266 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5bb3f5eb
2016-05-17 22:36:39,267 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5bb3f5eb
2016-05-17 22:36:39,267 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:39,267 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:39,269 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:39,269 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 6D44DCC180E14374BDC9E17C5219328E
2016-05-17 22:36:39,269 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_RESULTS"], id: 7F929A39C40B4F9881B32B980B26D39B
2016-05-17 22:36:39,270 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_RESULTS"]
2016-05-17 22:36:39,271 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:39,271 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:39,272 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D2C2754FDE514F49946A1F0B78F4960F, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:39,275 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:39,275 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:39,275 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524599055_0001_1_00_000000_0
2016-05-17 22:36:39,276 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524599055_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:39,276 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524599055_0001_1_00_000000_0
2016-05-17 22:36:39,277 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D2C2754FDE514F49946A1F0B78F4960F, mem on close (mb), free: 294, total: 388, max: 672
2016-05-17 22:36:39 Completed running task attempt: attempt_1463524599055_0001_1_00_000000_0
2016-05-17 22:36:39,278 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524599055_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:39,279 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599055_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D2C2754FDE514F49946A1F0B78F4960F, taskAttemptId=attempt_1463524599055_0001_1_00_000000_0, startTime=1463524599240, finishTime=1463524599279, timeTaken=39, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407371776, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524599272, Process_Duration=4, Process_End_Time=1463524599276, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:39,279 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524599055_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:39,279 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:39,279 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524599055_0001_1_00_000000_0
2016-05-17 22:36:39,280 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:39,280 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599055_0001_1][Event:TASK_FINISHED]: vertexName=D2C2754FDE514F49946A1F0B78F4960F, taskId=task_1463524599055_0001_1_00_000000, startTime=1463524599240, finishTime=1463524599280, timeTaken=40, status=SUCCEEDED, successfulAttemptID=attempt_1463524599055_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407371776, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524599272, Process_Duration=4, Process_End_Time=1463524599276, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:39,281 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524599055_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:39,281 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F] : 1
2016-05-17 22:36:39,281 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F]
2016-05-17 22:36:39,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407371776, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524599272, Process_Duration=4, Process_End_Time=1463524599276, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:39,281 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:39,281 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:39,282 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599055_0001_1][Event:VERTEX_FINISHED]: vertexName=D2C2754FDE514F49946A1F0B78F4960F, vertexId=vertex_1463524599055_0001_1_00, initRequestedTime=1463524599220, initedTime=1463524599231, startRequestedTime=1463524599231, startedTime=1463524599231, finishTime=1463524599281, timeTaken=50, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407371776, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524599272, Process_Duration=4, Process_End_Time=1463524599276, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524599240, firstTasksToStart=[ task_1463524599055_0001_1_00_000000 ], lastTaskFinishTime=1463524599279, lastTasksToFinish=[ task_1463524599055_0001_1_00_000000 ], minTaskDuration=39, maxTaskDuration=39, avgTaskDuration=39.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524599055_0001_1_00_000000 ], longestDurationTasks=[ task_1463524599055_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:39,282 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524599055_0001_00_000001
2016-05-17 22:36:39,282 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:39,283 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524599055_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:39,282 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:39,283 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599055_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524599055_0001_00_000001, stoppedTime=1463524599283, exitStatus=0
2016-05-17 22:36:39,283 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524599055_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:39,283 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:39,283 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524599055_0001_1
2016-05-17 22:36:39,283 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:39,284 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:39,286 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599055_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524599055_0001_1
2016-05-17 22:36:39,287 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 7F929A39C40B4F9881B32B980B26D39B for vertex: vertex_1463524599055_0001_1_00 [D2C2754FDE514F49946A1F0B78F4960F]
2016-05-17 22:36:39,292 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524599055_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:39,293 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524599055_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:39,294 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524599055_0001_00_000001, Exiting
2016-05-17 22:36:39,294 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524599055_0001_00_000001
2016-05-17 22:36:39,295 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524599055_0001_00_000001
2016-05-17 22:36:39,298 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524599055_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:39,298 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524599055_0001_00_000001 completed successfully
2016-05-17 22:36:39,298 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524599055_0001_1, queueSize=0
2016-05-17 22:36:39,298 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:39,299 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599055_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524599055_0001_1, startTime=1463524599220, finishTime=1463524599297, timeTaken=77, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407371776, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524599272, Process_Duration=4, Process_End_Time=1463524599276, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:39,299 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524599055_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:39,299 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524599055_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:39,299 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524599055_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:39 Completed Dag: dag_1463524599055_0001_1
2016-05-17 22:36:39,299 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524599055_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:39,300 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524599055_0001_1_post
2016-05-17 22:36:39,300 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:39,300 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:39,300 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524599055_0001_00_000001
2016-05-17 22:36:39,308 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/0_UNKNOWN_RESULTS, applicationId=application_1463524599055_0001
2016-05-17 22:36:39,309 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:39,310 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:39,310 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:39,310 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:39,310 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:39,310 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:39,311 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:39,311 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:39,311 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:39,311 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:39,311 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:39,311 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:39,312 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:39,315 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/084AD44E1F564E0BBAACAAD1FD6D71E2/D358190F9F2E470AAC1A89902EBFF0B0/.tez/application_1463524599055_0001
2016-05-17 22:36:39,315 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_RESULTS/_temporary
2016-05-17 22:36:39,315 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.279
2016-05-17 22:36:39,381 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_UNKNOWN_RESULTS
2016-05-17 22:36:39,401 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:39,401 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:39,429 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 22:36:39,438 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-17 22:36:39,439 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:39,439 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:39,439 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:39,452 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:39,453 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_ALL
2016-05-17 22:36:39,461 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:39,462 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:39,462 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_ALL"]
2016-05-17 22:36:39,462 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:39,462 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:39,462 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:39,463 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...blyplatform/0_UNKNOWN_ALL
2016-05-17 22:36:39,463 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:39,475 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:39,479 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:39,479 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:39,479 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:39,489 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/721A08B2CAF64FA1AB5FFB07CE4C9970/F37A40AEBEE942C8855EE5DA30D741E0/.tez/application_1463524599479_0001 doesn't exist and is created
2016-05-17 22:36:39,505 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:39,506 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:39,506 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/721A08B2CAF64FA1AB5FFB07CE4C9970/F37A40AEBEE942C8855EE5DA30D741E0/.tez/application_1463524599479_0001
2016-05-17 22:36:39,509 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524599479_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:39,518 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:39,519 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:39,520 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:39,520 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:39,521 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/721A08B2CAF64FA1AB5FFB07CE4C9970/F37A40AEBEE942C8855EE5DA30D741E0/.tez/application_1463524599479_0001/localmode-log-dir
2016-05-17 22:36:39,521 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/721A08B2CAF64FA1AB5FFB07CE4C9970/F37A40AEBEE942C8855EE5DA30D741E0/.tez/application_1463524599479_0001/localmode-log-dir/history.txt.appattempt_1463524599479_0001_000000, maxErrors=10
2016-05-17 22:36:39,521 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:39,521 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524599479_0001_000000, appSubmitTime=1463524599508, launchTime=1463524599509
2016-05-17 22:36:39,524 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:39,527 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:39,528 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:39,531 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:58927
2016-05-17 22:36:39,534 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:39,535 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:39,536 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524599479_0001_000000, startTime=1463524599536
2016-05-17 22:36:39,536 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:39,536 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:39,606 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:39,606 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:39,606 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/0_UNKNOWN_ALL, applicationId=application_1463524599479_0001, dagName=[721A08B2CAF64FA1AB5FFB07CE4C9970/F37A40AEBEE942C8855EE5DA30D741E0] each/(1/1) ...blyplatform/0_UNKNOWN_ALL
2016-05-17 22:36:39,622 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [721A08B2CAF64FA1AB5FFB07CE4C9970/F37A40AEBEE942C8855EE5DA30D741E0] each/(1/1) ...blyplatform/0_UNKNOWN_ALL
2016-05-17 22:36:39,623 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524599479_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/721A08B2CAF64FA1AB5FFB07CE4C9970/F37A40AEBEE942C8855EE5DA30D741E0/.tez/application_1463524599479_0001/localmode-log-dir/dag_1463524599479_0001_1.dot
2016-05-17 22:36:39,624 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/721A08B2CAF64FA1AB5FFB07CE4C9970/F37A40AEBEE942C8855EE5DA30D741E0/.tez/application_1463524599479_0001/localmode-log-dir/dag_1463524599479_0001_1-tez-dag.pb.txt
2016-05-17 22:36:39,629 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524599479_0001_1
2016-05-17 22:36:39,629 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:39,629 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [721A08B2CAF64FA1AB5FFB07CE4C9970/F37A40AEBEE942C8855EE5DA30D741E0] each/(1/1) ...blyplatform/0_UNKNOWN_ALL
2016-05-17 22:36:39 Running Dag: dag_1463524599479_0001_1
2016-05-17 22:36:39,642 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599479_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524599479_0001_1, submitTime=1463524599622
2016-05-17 22:36:39,642 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:39,643 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F4B54EB85CC94C1692F35C62E56E840C
2016-05-17 22:36:39,643 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:39,643 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599479_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524599479_0001_1, initTime=1463524599642
2016-05-17 22:36:39,643 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:39,643 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524599479_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:39,644 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599479_0001_1][Event:DAG_STARTED]: dagID=dag_1463524599479_0001_1, startTime=1463524599643
2016-05-17 22:36:39,644 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:39,644 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/0_UNKNOWN_ALL, applicationId=application_1463524599479_0001, dagName=[721A08B2CAF64FA1AB5FFB07CE4C9970/F37A40AEBEE942C8855EE5DA30D741E0] each/(1/1) ...blyplatform/0_UNKNOWN_ALL
2016-05-17 22:36:39,644 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:39,644 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524599479_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:39,644 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F4B54EB85CC94C1692F35C62E56E840C : {3CDE5EBDD0634FD8B81A408A1DC58EAD={InputName=3CDE5EBDD0634FD8B81A408A1DC58EAD}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:39,644 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 3CDE5EBDD0634FD8B81A408A1DC58EAD, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:39,644 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C]
2016-05-17 22:36:39,645 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C]
2016-05-17 22:36:39,645 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C]
2016-05-17 22:36:39,645 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C]. Starting root input initializers: 1
2016-05-17 22:36:39,645 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:39,646 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 3CDE5EBDD0634FD8B81A408A1DC58EAD on vertex vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C]
2016-05-17 22:36:39,647 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:39,648 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 3CDE5EBDD0634FD8B81A408A1DC58EAD asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:39,648 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:39,648 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:39,649 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:39,650 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 3CDE5EBDD0634FD8B81A408A1DC58EAD on vertex vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C]
2016-05-17 22:36:39,650 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:39,650 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {3CDE5EBDD0634FD8B81A408A1DC58EAD=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:39,650 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C] parallelism set to 1
2016-05-17 22:36:39,650 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:39,651 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599479_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524599479_0001_1_00, eventCount=1
2016-05-17 22:36:39,651 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C]
2016-05-17 22:36:39,651 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=3094A6E32FEE46959BBE708DC58DAC34, vertexId=vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:39,651 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=3094A6E32FEE46959BBE708DC58DAC34, vertexId=vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C]
2016-05-17 22:36:39,651 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:39,651 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:39,653 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=3094A6E32FEE46959BBE708DC58DAC34, vertexName=F4B54EB85CC94C1692F35C62E56E840C, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:39,653 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=3094A6E32FEE46959BBE708DC58DAC34, vertexId=vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C]
2016-05-17 22:36:39,654 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:39,654 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599479_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F4B54EB85CC94C1692F35C62E56E840C, vertexId=vertex_1463524599479_0001_1_00, initRequestedTime=1463524599644, initedTime=1463524599654, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:39,655 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C] with distanceFromRoot: 0
2016-05-17 22:36:39,655 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:39,655 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F4B54EB85CC94C1692F35C62E56E840C
2016-05-17 22:36:39,655 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C]
2016-05-17 22:36:39,655 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:39,655 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599479_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524599479_0001_1_00, startRequestedTime=1463524599655, startedTime=1463524599655
2016-05-17 22:36:39,655 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:39,656 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:39,656 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599479_0001_1][Event:TASK_STARTED]: vertexName=F4B54EB85CC94C1692F35C62E56E840C, taskId=task_1463524599479_0001_1_00_000000, scheduledTime=1463524599656, launchTime=1463524599656
2016-05-17 22:36:39,656 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524599479_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:39,657 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C]
2016-05-17 22:36:39,657 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [721A08B2CAF64FA1AB5FFB07CE4C9970/F37A40AEBEE942C8855EE5DA30D741E0] each/(1/1) ...blyplatform/0_UNKNOWN_ALL, VertexName: F4B54EB85CC94C1692F35C62E56E840C, VertexParallelism: 1, TaskAttemptID:attempt_1463524599479_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3CDE5EBDD0634FD8B81A408A1DC58EAD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=3094A6E32FEE46959BBE708DC58DAC34, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:39,657 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524599479_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:39,657 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:39,658 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:39,659 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:39,659 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:39,659 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524599479_0001_00_000001
2016-05-17 22:36:39,659 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524599479_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:39,661 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524599479_0001_00_000001
2016-05-17 22:36:39,661 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:39,662 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524599479_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:39,662 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:39,663 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599479_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524599479_0001_00_000001, launchTime=1463524599662
2016-05-17 22:36:39,663 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524599479_0001_00_000001
2016-05-17 22:36:39,664 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524599479_0001_1_00_000000_0] to container: [container_1463524599479_0001_00_000001]
2016-05-17 22:36:39,664 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524599479_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:39,664 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524599479_0001_1_00_000000_0] started. Is using containerId: [container_1463524599479_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:39,664 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599479_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F4B54EB85CC94C1692F35C62E56E840C, taskAttemptId=attempt_1463524599479_0001_1_00_000000_0, startTime=1463524599664, containerId=container_1463524599479_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524599479_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:39,664 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524599479_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:39,665 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524599479_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:39,665 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:39,665 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524599479_0001, with dag id: dag_1463524599479_0001_1
2016-05-17 22:36:39,665 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524599479_0001_00_000001 given task: attempt_1463524599479_0001_1_00_000000_0
2016-05-17 22:36:39,665 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524599479_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524599479_0001_1_00_000000_0
2016-05-17 22:36:39 Starting to run new task attempt: attempt_1463524599479_0001_1_00_000000_0
2016-05-17 22:36:39,666 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524599479_0001_1_00_000000_0
2016-05-17 22:36:39,666 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:39,666 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:39,666 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:39,666 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:39,666 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:39,667 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:39,667 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [721A08B2CAF64FA1AB5FFB07CE4C9970/F37A40AEBEE942C8855EE5DA30D741E0] each/(1/1) ...blyplatform/0_UNKNOWN_ALL, VertexName: F4B54EB85CC94C1692F35C62E56E840C, VertexParallelism: 1, TaskAttemptID:attempt_1463524599479_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3CDE5EBDD0634FD8B81A408A1DC58EAD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=3094A6E32FEE46959BBE708DC58DAC34, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:39,667 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:39,668 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524599479_0001_1_00_000000_0
2016-05-17 22:36:39,668 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:39,668 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:39,668 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:39,669 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=3CDE5EBDD0634FD8B81A408A1DC58EAD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:39,669 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:39,670 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524599479_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:39,670 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 3CDE5EBDD0634FD8B81A408A1DC58EAD
2016-05-17 22:36:39,670 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 3CDE5EBDD0634FD8B81A408A1DC58EAD
2016-05-17 22:36:39,670 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=3094A6E32FEE46959BBE708DC58DAC34, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:39,672 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:39,672 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 3094A6E32FEE46959BBE708DC58DAC34
2016-05-17 22:36:39,673 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:39,673 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 3094A6E32FEE46959BBE708DC58DAC34
2016-05-17 22:36:39,672 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:39,675 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:39,676 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 3CDE5EBDD0634FD8B81A408A1DC58EAD
2016-05-17 22:36:39,676 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 3CDE5EBDD0634FD8B81A408A1DC58EAD
2016-05-17 22:36:39,680 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F4B54EB85CC94C1692F35C62E56E840C, ordinal: 0
2016-05-17 22:36:39,680 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F4B54EB85CC94C1692F35C62E56E840C, mem on start (mb), free: 220, total: 388, max: 672
2016-05-17 22:36:39,681 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:39,681 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:39,681 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:39,687 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:39,687 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 3094A6E32FEE46959BBE708DC58DAC34
2016-05-17 22:36:39,687 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:39,687 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:39,687 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:39,687 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:39,687 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:39,688 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:39,688 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:39,688 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:39,688 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 3CDE5EBDD0634FD8B81A408A1DC58EAD, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:39,688 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 3094A6E32FEE46959BBE708DC58DAC34, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:39,688 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:39,688 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 3CDE5EBDD0634FD8B81A408A1DC58EAD being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:39,688 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:39,689 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:39,688 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 3CDE5EBDD0634FD8B81A408A1DC58EAD
2016-05-17 22:36:39,689 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:39,689 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 3CDE5EBDD0634FD8B81A408A1DC58EAD
2016-05-17 22:36:39,689 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:39,690 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524599479_0001_1_00_000000_0
2016-05-17 22:36:39,690 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:39,690 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3ad1a015
2016-05-17 22:36:39,691 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3ad1a015
2016-05-17 22:36:39,691 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:39,692 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:39,693 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:39,694 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 3CDE5EBDD0634FD8B81A408A1DC58EAD
2016-05-17 22:36:39,694 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_ALL"], id: 3094A6E32FEE46959BBE708DC58DAC34
2016-05-17 22:36:39,694 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_ALL"]
2016-05-17 22:36:39,694 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:39,695 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:39,695 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F4B54EB85CC94C1692F35C62E56E840C, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:39,698 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:39,698 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:39,699 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524599479_0001_1_00_000000_0
2016-05-17 22:36:39,699 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524599479_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:39,699 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524599479_0001_1_00_000000_0
2016-05-17 22:36:39,700 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F4B54EB85CC94C1692F35C62E56E840C, mem on close (mb), free: 218, total: 388, max: 672
2016-05-17 22:36:39 Completed running task attempt: attempt_1463524599479_0001_1_00_000000_0
2016-05-17 22:36:39,704 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524599479_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:39,705 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407371776, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524599695, Process_Duration=4, Process_End_Time=1463524599699, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:39,705 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599479_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F4B54EB85CC94C1692F35C62E56E840C, taskAttemptId=attempt_1463524599479_0001_1_00_000000_0, startTime=1463524599664, finishTime=1463524599705, timeTaken=41, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407371776, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524599695, Process_Duration=4, Process_End_Time=1463524599699, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:39,705 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:39,706 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:39,706 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:39,706 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524599479_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:39,706 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524599479_0001_1_00_000000_0
2016-05-17 22:36:39,706 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:39,707 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599479_0001_1][Event:TASK_FINISHED]: vertexName=F4B54EB85CC94C1692F35C62E56E840C, taskId=task_1463524599479_0001_1_00_000000, startTime=1463524599664, finishTime=1463524599706, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1463524599479_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407371776, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524599695, Process_Duration=4, Process_End_Time=1463524599699, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:39,707 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524599479_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:39,707 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C] : 1
2016-05-17 22:36:39,707 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C]
2016-05-17 22:36:39,707 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:39,708 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:39,708 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599479_0001_1][Event:VERTEX_FINISHED]: vertexName=F4B54EB85CC94C1692F35C62E56E840C, vertexId=vertex_1463524599479_0001_1_00, initRequestedTime=1463524599644, initedTime=1463524599654, startRequestedTime=1463524599655, startedTime=1463524599655, finishTime=1463524599707, timeTaken=52, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407371776, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524599695, Process_Duration=4, Process_End_Time=1463524599699, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524599664, firstTasksToStart=[ task_1463524599479_0001_1_00_000000 ], lastTaskFinishTime=1463524599705, lastTasksToFinish=[ task_1463524599479_0001_1_00_000000 ], minTaskDuration=41, maxTaskDuration=41, avgTaskDuration=41.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524599479_0001_1_00_000000 ], longestDurationTasks=[ task_1463524599479_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:39,708 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:39,708 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524599479_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:39,709 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:39,709 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524599479_0001_1
2016-05-17 22:36:39,709 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599479_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524599479_0001_1
2016-05-17 22:36:39,710 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 3094A6E32FEE46959BBE708DC58DAC34 for vertex: vertex_1463524599479_0001_1_00 [F4B54EB85CC94C1692F35C62E56E840C]
2016-05-17 22:36:39,710 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524599479_0001_00_000001
2016-05-17 22:36:39,718 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524599479_0001_1, queueSize=0
2016-05-17 22:36:39,718 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:39,718 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599479_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524599479_0001_1, startTime=1463524599643, finishTime=1463524599717, timeTaken=74, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407371776, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524599695, Process_Duration=4, Process_End_Time=1463524599699, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:39,718 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524599479_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:39,718 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524599479_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:39,718 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:39,719 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599479_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524599479_0001_00_000001, stoppedTime=1463524599718, exitStatus=0
2016-05-17 22:36:39,719 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524599479_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:39 Completed Dag: dag_1463524599479_0001_1
2016-05-17 22:36:39,720 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524599479_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:39,720 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524599479_0001_1_post
2016-05-17 22:36:39,720 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:39,721 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:39,721 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524599479_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:36:39,721 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524599479_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:39,721 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524599479_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:39,721 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524599479_0001_00_000001, Exiting
2016-05-17 22:36:39,721 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524599479_0001_00_000001
2016-05-17 22:36:39,721 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524599479_0001_00_000001
2016-05-17 22:36:39,722 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524599479_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:36:39,722 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524599479_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:39,723 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524599479_0001_00_000001 completed successfully
2016-05-17 22:36:39,723 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524599479_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:36:39,725 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/0_UNKNOWN_ALL, applicationId=application_1463524599479_0001
2016-05-17 22:36:39,727 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:39,727 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:39,727 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:39,727 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:39,727 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:39,727 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:39,728 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:39,728 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:39,728 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:39,729 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:39,729 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:39,729 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:39,730 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:39,733 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/721A08B2CAF64FA1AB5FFB07CE4C9970/F37A40AEBEE942C8855EE5DA30D741E0/.tez/application_1463524599479_0001
2016-05-17 22:36:39,733 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_ALL/_temporary
2016-05-17 22:36:39,734 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.272
2016-05-17 22:36:39,798 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_UNKNOWN_ALL
2016-05-17 22:36:39,818 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:39,818 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:39,848 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 22:36:39,853 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 22:36:39,854 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:39,854 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:39,854 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:39,868 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:39,868 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_REPLACE
2016-05-17 22:36:39,878 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:39,878 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:39,878 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_REPLACE"]
2016-05-17 22:36:39,878 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:39,878 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:39,878 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:39,881 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...latform/0_UNKNOWN_REPLACE
2016-05-17 22:36:39,881 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:39,900 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:39,905 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:39,905 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:39,906 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:39,917 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/76C8FA5678894C8FAB0506207561AC8D/C379E11688054899A8DA61604B5F6FEC/.tez/application_1463524599905_0001 doesn't exist and is created
2016-05-17 22:36:39,937 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:39,937 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:39,937 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/76C8FA5678894C8FAB0506207561AC8D/C379E11688054899A8DA61604B5F6FEC/.tez/application_1463524599905_0001
2016-05-17 22:36:39,943 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524599905_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:39,959 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:39,960 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:39,962 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:39,962 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:39,963 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/76C8FA5678894C8FAB0506207561AC8D/C379E11688054899A8DA61604B5F6FEC/.tez/application_1463524599905_0001/localmode-log-dir
2016-05-17 22:36:39,963 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/76C8FA5678894C8FAB0506207561AC8D/C379E11688054899A8DA61604B5F6FEC/.tez/application_1463524599905_0001/localmode-log-dir/history.txt.appattempt_1463524599905_0001_000000, maxErrors=10
2016-05-17 22:36:39,963 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:39,963 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524599905_0001_000000, appSubmitTime=1463524599942, launchTime=1463524599942
2016-05-17 22:36:39,968 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:39,969 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:39,971 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:39,971 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:39,974 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:53994
2016-05-17 22:36:39,975 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:39,977 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524599905_0001_000000, startTime=1463524599977
2016-05-17 22:36:39,977 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:39,977 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:40,037 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:40,037 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:40,037 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/0_UNKNOWN_REPLACE, applicationId=application_1463524599905_0001, dagName=[76C8FA5678894C8FAB0506207561AC8D/C379E11688054899A8DA61604B5F6FEC] each/(1/1) ...latform/0_UNKNOWN_REPLACE
2016-05-17 22:36:40,052 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [76C8FA5678894C8FAB0506207561AC8D/C379E11688054899A8DA61604B5F6FEC] each/(1/1) ...latform/0_UNKNOWN_REPLACE
2016-05-17 22:36:40,054 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524599905_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/76C8FA5678894C8FAB0506207561AC8D/C379E11688054899A8DA61604B5F6FEC/.tez/application_1463524599905_0001/localmode-log-dir/dag_1463524599905_0001_1.dot
2016-05-17 22:36:40,055 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/76C8FA5678894C8FAB0506207561AC8D/C379E11688054899A8DA61604B5F6FEC/.tez/application_1463524599905_0001/localmode-log-dir/dag_1463524599905_0001_1-tez-dag.pb.txt
2016-05-17 22:36:40,060 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524599905_0001_1
2016-05-17 22:36:40,060 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:40,060 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [76C8FA5678894C8FAB0506207561AC8D/C379E11688054899A8DA61604B5F6FEC] each/(1/1) ...latform/0_UNKNOWN_REPLACE
2016-05-17 22:36:40 Running Dag: dag_1463524599905_0001_1
2016-05-17 22:36:40,072 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:40,073 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599905_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524599905_0001_1, submitTime=1463524600052
2016-05-17 22:36:40,073 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A6C428415FB54C7E9CEF46E244719AA9
2016-05-17 22:36:40,073 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:40,074 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:40,074 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599905_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524599905_0001_1, initTime=1463524600073
2016-05-17 22:36:40,074 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524599905_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:40,074 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599905_0001_1][Event:DAG_STARTED]: dagID=dag_1463524599905_0001_1, startTime=1463524600074
2016-05-17 22:36:40,075 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:40,075 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/0_UNKNOWN_REPLACE, applicationId=application_1463524599905_0001, dagName=[76C8FA5678894C8FAB0506207561AC8D/C379E11688054899A8DA61604B5F6FEC] each/(1/1) ...latform/0_UNKNOWN_REPLACE
2016-05-17 22:36:40,074 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:40,075 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524599905_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:40,075 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A6C428415FB54C7E9CEF46E244719AA9 : {2066404522064BD2A6AB351F9781C15F={InputName=2066404522064BD2A6AB351F9781C15F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:40,075 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 2066404522064BD2A6AB351F9781C15F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:40,075 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9]
2016-05-17 22:36:40,075 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9]
2016-05-17 22:36:40,075 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9]
2016-05-17 22:36:40,076 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9]. Starting root input initializers: 1
2016-05-17 22:36:40,076 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:40,076 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 2066404522064BD2A6AB351F9781C15F on vertex vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9]
2016-05-17 22:36:40,078 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:40,078 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 2066404522064BD2A6AB351F9781C15F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:40,079 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:40,079 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:40,080 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:40,080 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 2066404522064BD2A6AB351F9781C15F on vertex vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9]
2016-05-17 22:36:40,081 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:40,081 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {2066404522064BD2A6AB351F9781C15F=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:40,081 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9] parallelism set to 1
2016-05-17 22:36:40,081 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:40,081 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599905_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524599905_0001_1_00, eventCount=1
2016-05-17 22:36:40,081 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9]
2016-05-17 22:36:40,082 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=DF751B63F6CF424C86F3F130DEDC0399, vertexId=vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:40,082 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=DF751B63F6CF424C86F3F130DEDC0399, vertexId=vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9]
2016-05-17 22:36:40,084 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=DF751B63F6CF424C86F3F130DEDC0399, vertexName=A6C428415FB54C7E9CEF46E244719AA9, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:40,084 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=DF751B63F6CF424C86F3F130DEDC0399, vertexId=vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9]
2016-05-17 22:36:40,085 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599905_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A6C428415FB54C7E9CEF46E244719AA9, vertexId=vertex_1463524599905_0001_1_00, initRequestedTime=1463524600075, initedTime=1463524600085, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:40,085 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9] with distanceFromRoot: 0
2016-05-17 22:36:40,085 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:40,086 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:40,086 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A6C428415FB54C7E9CEF46E244719AA9
2016-05-17 22:36:40,086 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9]
2016-05-17 22:36:40,086 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:40,087 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599905_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524599905_0001_1_00, startRequestedTime=1463524600086, startedTime=1463524600086
2016-05-17 22:36:40,087 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:40,087 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:40,088 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599905_0001_1][Event:TASK_STARTED]: vertexName=A6C428415FB54C7E9CEF46E244719AA9, taskId=task_1463524599905_0001_1_00_000000, scheduledTime=1463524600087, launchTime=1463524600087
2016-05-17 22:36:40,088 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524599905_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:40,088 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9]
2016-05-17 22:36:40,089 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [76C8FA5678894C8FAB0506207561AC8D/C379E11688054899A8DA61604B5F6FEC] each/(1/1) ...latform/0_UNKNOWN_REPLACE, VertexName: A6C428415FB54C7E9CEF46E244719AA9, VertexParallelism: 1, TaskAttemptID:attempt_1463524599905_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2066404522064BD2A6AB351F9781C15F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DF751B63F6CF424C86F3F130DEDC0399, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:40,089 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524599905_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:40,089 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:40,090 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:40,090 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:40,090 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:40,091 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524599905_0001_00_000001
2016-05-17 22:36:40,091 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524599905_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:40,092 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524599905_0001_00_000001
2016-05-17 22:36:40,092 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:40,093 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524599905_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:40,093 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599905_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524599905_0001_00_000001, launchTime=1463524600093
2016-05-17 22:36:40,093 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:40,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524599905_0001, with dag id: dag_1463524599905_0001_1
2016-05-17 22:36:40,094 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524599905_0001_00_000001
2016-05-17 22:36:40,095 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524599905_0001_1_00_000000_0] to container: [container_1463524599905_0001_00_000001]
2016-05-17 22:36:40,095 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524599905_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:40,095 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524599905_0001_00_000001 given task: attempt_1463524599905_0001_1_00_000000_0
2016-05-17 22:36:40,095 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524599905_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524599905_0001_1_00_000000_0
2016-05-17 22:36:40 Starting to run new task attempt: attempt_1463524599905_0001_1_00_000000_0
2016-05-17 22:36:40,096 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524599905_0001_1_00_000000_0
2016-05-17 22:36:40,096 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:40,096 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:40,096 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:40,096 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:40,096 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:40,097 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:40,097 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [76C8FA5678894C8FAB0506207561AC8D/C379E11688054899A8DA61604B5F6FEC] each/(1/1) ...latform/0_UNKNOWN_REPLACE, VertexName: A6C428415FB54C7E9CEF46E244719AA9, VertexParallelism: 1, TaskAttemptID:attempt_1463524599905_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2066404522064BD2A6AB351F9781C15F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DF751B63F6CF424C86F3F130DEDC0399, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:40,097 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:40,098 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524599905_0001_1_00_000000_0
2016-05-17 22:36:40,098 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:40,098 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524599905_0001_1_00_000000_0] started. Is using containerId: [container_1463524599905_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:40,098 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:40,099 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524599905_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:40,099 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599905_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A6C428415FB54C7E9CEF46E244719AA9, taskAttemptId=attempt_1463524599905_0001_1_00_000000_0, startTime=1463524600097, containerId=container_1463524599905_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524599905_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:40,100 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524599905_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:40,100 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:40,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:40,100 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524599905_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:40,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=2066404522064BD2A6AB351F9781C15F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:40,101 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:40,101 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 2066404522064BD2A6AB351F9781C15F
2016-05-17 22:36:40,101 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 2066404522064BD2A6AB351F9781C15F
2016-05-17 22:36:40,102 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DF751B63F6CF424C86F3F130DEDC0399, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:40,103 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:40,103 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:40,103 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 2066404522064BD2A6AB351F9781C15F
2016-05-17 22:36:40,103 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 2066404522064BD2A6AB351F9781C15F
2016-05-17 22:36:40,104 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:40,104 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DF751B63F6CF424C86F3F130DEDC0399
2016-05-17 22:36:40,104 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:40,104 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: DF751B63F6CF424C86F3F130DEDC0399
2016-05-17 22:36:40,109 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A6C428415FB54C7E9CEF46E244719AA9, ordinal: 0
2016-05-17 22:36:40,109 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A6C428415FB54C7E9CEF46E244719AA9, mem on start (mb), free: 146, total: 388, max: 672
2016-05-17 22:36:40,109 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:40,109 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:40,109 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:40,115 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:40,115 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DF751B63F6CF424C86F3F130DEDC0399
2016-05-17 22:36:40,115 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:40,115 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:40,115 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:40,115 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:40,115 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:40,115 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:40,116 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:40,116 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:40,116 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 2066404522064BD2A6AB351F9781C15F, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:40,116 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DF751B63F6CF424C86F3F130DEDC0399, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:40,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:40,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 2066404522064BD2A6AB351F9781C15F being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:40,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:40,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 2066404522064BD2A6AB351F9781C15F
2016-05-17 22:36:40,116 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:40,117 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 2066404522064BD2A6AB351F9781C15F
2016-05-17 22:36:40,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:40,117 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:40,117 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524599905_0001_1_00_000000_0
2016-05-17 22:36:40,117 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:40,118 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@475137e1
2016-05-17 22:36:40,119 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@475137e1
2016-05-17 22:36:40,119 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:40,119 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:40,120 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:40,120 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 2066404522064BD2A6AB351F9781C15F
2016-05-17 22:36:40,121 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_REPLACE"], id: DF751B63F6CF424C86F3F130DEDC0399
2016-05-17 22:36:40,121 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_REPLACE"]
2016-05-17 22:36:40,121 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:40,121 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:40,121 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A6C428415FB54C7E9CEF46E244719AA9, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:40,122 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:40,122 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:40,123 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:40,124 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:40,125 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524599905_0001_1_00_000000_0
2016-05-17 22:36:40,125 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524599905_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:40,126 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524599905_0001_1_00_000000_0
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:40,128 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599905_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A6C428415FB54C7E9CEF46E244719AA9, taskAttemptId=attempt_1463524599905_0001_1_00_000000_0, startTime=1463524600097, finishTime=1463524600127, timeTaken=30, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407371776, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524600121, Process_Duration=5, Process_End_Time=1463524600126, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:36:40,128 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524599905_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:36:40,128 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524599905_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:36:40,128 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599905_0001_1][Event:TASK_FINISHED]: vertexName=A6C428415FB54C7E9CEF46E244719AA9, taskId=task_1463524599905_0001_1_00_000000, startTime=1463524600097, finishTime=1463524600128, timeTaken=31, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
], counters=Counters: 0
2016-05-17 22:36:40,128 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524599905_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:36:40,129 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:40,129 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9] : 1
2016-05-17 22:36:40,129 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9] because task failed: task_1463524599905_0001_1_00_000000
2016-05-17 22:36:40,129 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:36:40,129 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:36:40,129 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9]
2016-05-17 22:36:40,129 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=DF751B63F6CF424C86F3F130DEDC0399, vertexId=vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9]
2016-05-17 22:36:40,130 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599905_0001_1][Event:VERTEX_FINISHED]: vertexName=A6C428415FB54C7E9CEF46E244719AA9, vertexId=vertex_1463524599905_0001_1_00, initRequestedTime=1463524600075, initedTime=1463524600085, startRequestedTime=1463524600086, startedTime=1463524600086, finishTime=1463524600130, timeTaken=44, status=FAILED, diagnostics=Task failed, taskId=task_1463524599905_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524600097, firstTasksToStart=[ task_1463524599905_0001_1_00_000000 ], lastTaskFinishTime=1463524600127, lastTasksToFinish=[ task_1463524599905_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:36:40,131 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:36:40,131 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:40,131 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:40,132 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:40,132 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599905_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524599905_0001_00_000001, stoppedTime=1463524600132, exitStatus=0
2016-05-17 22:36:40,133 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524599905_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:40,133 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524599905_0001_1_00_000000_0 failed: true container: container_1463524599905_0001_00_000001 numFailedTAs: 0
2016-05-17 22:36:40,133 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:36:40,133 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:36:40,133 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524599905_0001_1
2016-05-17 22:36:40,133 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: DF751B63F6CF424C86F3F130DEDC0399 for vertex: vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9]
2016-05-17 22:36:40,134 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524599905_0001_1, queueSize=0
2016-05-17 22:36:40,134 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524599905_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524599905_0001_1, startTime=1463524600074, finishTime=1463524600134, timeTaken=60, status=FAILED, diagnostics=Vertex failed, vertexName=A6C428415FB54C7E9CEF46E244719AA9, vertexId=vertex_1463524599905_0001_1_00, diagnostics=[Task failed, taskId=task_1463524599905_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524599905_0001_1_00 [A6C428415FB54C7E9CEF46E244719AA9] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:36:40,134 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524599905_0001_1 finished with state: FAILED
2016-05-17 22:36:40,134 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524599905_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:40,135 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:40,135 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:40,135 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524599905_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:36:40 Completed Dag: dag_1463524599905_0001_1
2016-05-17 22:36:40,136 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524599905_0001_1, dagState=FAILED
2016-05-17 22:36:40,136 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524599905_0001_1_post
2016-05-17 22:36:40,136 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:40,136 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:40,136 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524599905_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:36:40,136 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 19 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=407371776, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524600121, Process_Duration=5, Process_End_Time=1463524600126, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:36:40,136 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A6C428415FB54C7E9CEF46E244719AA9, mem on close (mb), free: 142, total: 388, max: 672
2016-05-17 22:36:40,136 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=A6C428415FB54C7E9CEF46E244719AA9, index=0
2016-05-17 22:36:40,136 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=A6C428415FB54C7E9CEF46E244719AA9, sourceVertex=2066404522064BD2A6AB351F9781C15F
2016-05-17 22:36:40,137 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=A6C428415FB54C7E9CEF46E244719AA9, sourceVertex=DF751B63F6CF424C86F3F130DEDC0399
2016-05-17 22:36:40,137 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524599905_0001_00_000001
2016-05-17 22:36:40,137 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524599905_0001_00_000001
2016-05-17 22:36:40,150 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524599905_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:36:40,151 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524599905_0001_00_000001 completed but with errors
2016-05-17 22:36:40,151 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524599905_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:36:40,151 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524599905_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:36:40,151 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524599905_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:36:40,152 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/0_UNKNOWN_REPLACE, applicationId=application_1463524599905_0001
2016-05-17 22:36:40,154 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:40,154 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:40,154 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:40,154 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:40,154 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:40,155 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:40,155 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:40,155 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:40,156 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:40,156 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:40,156 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:40,156 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:40,157 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:40,160 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/76C8FA5678894C8FAB0506207561AC8D/C379E11688054899A8DA61604B5F6FEC/.tez/application_1463524599905_0001
2016-05-17 22:36:40,160 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping all jobs
2016-05-17 22:36:40,161 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping: (1/1) ...latform/0_UNKNOWN_REPLACE
2016-05-17 22:36:40,162 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopped all jobs
2016-05-17 22:36:40,162 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_REPLACE/_temporary
2016-05-17 22:36:40,162 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.284
2016-05-17 22:36:40,163 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_UNKNOWN_REPLACE
2016-05-17 22:36:40,183 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:40,183 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:40,194 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:40,194 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:40,224 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 22:36:40,233 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-17 22:36:40,234 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:40,234 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:40,234 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:40,247 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:40,248 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_SWAP
2016-05-17 22:36:40,270 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:40,271 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:40,271 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_SWAP"]
2016-05-17 22:36:40,272 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:40,272 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:40,272 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:40,273 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...lyplatform/0_UNKNOWN_SWAP
2016-05-17 22:36:40,273 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:40,286 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:40,291 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:40,291 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:40,292 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:40,303 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/75EB0E8F8C1D426FAE1673C9F1827659/D73FAC7AD9F845A0B50B097DA746432A/.tez/application_1463524600291_0001 doesn't exist and is created
2016-05-17 22:36:40,318 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:40,318 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:40,318 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/75EB0E8F8C1D426FAE1673C9F1827659/D73FAC7AD9F845A0B50B097DA746432A/.tez/application_1463524600291_0001
2016-05-17 22:36:40,322 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524600291_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:40,332 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:40,333 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:40,334 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:40,334 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:40,334 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/75EB0E8F8C1D426FAE1673C9F1827659/D73FAC7AD9F845A0B50B097DA746432A/.tez/application_1463524600291_0001/localmode-log-dir
2016-05-17 22:36:40,334 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/75EB0E8F8C1D426FAE1673C9F1827659/D73FAC7AD9F845A0B50B097DA746432A/.tez/application_1463524600291_0001/localmode-log-dir/history.txt.appattempt_1463524600291_0001_000000, maxErrors=10
2016-05-17 22:36:40,335 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:40,335 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524600291_0001_000000, appSubmitTime=1463524600321, launchTime=1463524600321
2016-05-17 22:36:40,340 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:36622
2016-05-17 22:36:40,344 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:40,346 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:40,347 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:40,349 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:40,349 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:40,351 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524600291_0001_000000, startTime=1463524600351
2016-05-17 22:36:40,351 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:40,351 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:40,418 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:40,418 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:40,419 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/0_UNKNOWN_SWAP, applicationId=application_1463524600291_0001, dagName=[75EB0E8F8C1D426FAE1673C9F1827659/D73FAC7AD9F845A0B50B097DA746432A] each/(1/1) ...lyplatform/0_UNKNOWN_SWAP
2016-05-17 22:36:40,433 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [75EB0E8F8C1D426FAE1673C9F1827659/D73FAC7AD9F845A0B50B097DA746432A] each/(1/1) ...lyplatform/0_UNKNOWN_SWAP
2016-05-17 22:36:40,434 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524600291_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/75EB0E8F8C1D426FAE1673C9F1827659/D73FAC7AD9F845A0B50B097DA746432A/.tez/application_1463524600291_0001/localmode-log-dir/dag_1463524600291_0001_1.dot
2016-05-17 22:36:40,435 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/75EB0E8F8C1D426FAE1673C9F1827659/D73FAC7AD9F845A0B50B097DA746432A/.tez/application_1463524600291_0001/localmode-log-dir/dag_1463524600291_0001_1-tez-dag.pb.txt
2016-05-17 22:36:40,438 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524600291_0001_1
2016-05-17 22:36:40,438 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:40,439 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [75EB0E8F8C1D426FAE1673C9F1827659/D73FAC7AD9F845A0B50B097DA746432A] each/(1/1) ...lyplatform/0_UNKNOWN_SWAP
2016-05-17 22:36:40 Running Dag: dag_1463524600291_0001_1
2016-05-17 22:36:40,451 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600291_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524600291_0001_1, submitTime=1463524600433
2016-05-17 22:36:40,451 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:40,451 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F9FDDBA2BFD64DF8B8415866318968D2
2016-05-17 22:36:40,452 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:40,453 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600291_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524600291_0001_1, initTime=1463524600451
2016-05-17 22:36:40,453 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:40,453 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524600291_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:40,453 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600291_0001_1][Event:DAG_STARTED]: dagID=dag_1463524600291_0001_1, startTime=1463524600453
2016-05-17 22:36:40,453 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:40,453 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524600291_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:40,453 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/0_UNKNOWN_SWAP, applicationId=application_1463524600291_0001, dagName=[75EB0E8F8C1D426FAE1673C9F1827659/D73FAC7AD9F845A0B50B097DA746432A] each/(1/1) ...lyplatform/0_UNKNOWN_SWAP
2016-05-17 22:36:40,454 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:40,454 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F9FDDBA2BFD64DF8B8415866318968D2 : {E3F76FB2565442C9A69716E2D98205E1={InputName=E3F76FB2565442C9A69716E2D98205E1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:40,454 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: E3F76FB2565442C9A69716E2D98205E1, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:40,454 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2]
2016-05-17 22:36:40,454 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2]
2016-05-17 22:36:40,454 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2]
2016-05-17 22:36:40,454 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2]. Starting root input initializers: 1
2016-05-17 22:36:40,455 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:40,455 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: E3F76FB2565442C9A69716E2D98205E1 on vertex vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2]
2016-05-17 22:36:40,457 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:40,457 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input E3F76FB2565442C9A69716E2D98205E1 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:40,458 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:40,458 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:40,459 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:40,460 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: E3F76FB2565442C9A69716E2D98205E1 on vertex vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2]
2016-05-17 22:36:40,460 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:40,460 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {E3F76FB2565442C9A69716E2D98205E1=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:40,460 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2] parallelism set to 1
2016-05-17 22:36:40,460 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:40,461 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600291_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524600291_0001_1_00, eventCount=1
2016-05-17 22:36:40,461 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2]
2016-05-17 22:36:40,461 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=FFF09557E61D45A68269F4A4EACC2037, vertexId=vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:40,461 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=FFF09557E61D45A68269F4A4EACC2037, vertexId=vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2]
2016-05-17 22:36:40,462 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=FFF09557E61D45A68269F4A4EACC2037, vertexName=F9FDDBA2BFD64DF8B8415866318968D2, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:40,463 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=FFF09557E61D45A68269F4A4EACC2037, vertexId=vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2]
2016-05-17 22:36:40,464 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:40,464 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600291_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F9FDDBA2BFD64DF8B8415866318968D2, vertexId=vertex_1463524600291_0001_1_00, initRequestedTime=1463524600454, initedTime=1463524600464, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:40,464 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2] with distanceFromRoot: 0
2016-05-17 22:36:40,464 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:40,465 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F9FDDBA2BFD64DF8B8415866318968D2
2016-05-17 22:36:40,465 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2]
2016-05-17 22:36:40,465 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:40,465 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600291_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524600291_0001_1_00, startRequestedTime=1463524600465, startedTime=1463524600465
2016-05-17 22:36:40,466 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:40,466 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:40,466 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600291_0001_1][Event:TASK_STARTED]: vertexName=F9FDDBA2BFD64DF8B8415866318968D2, taskId=task_1463524600291_0001_1_00_000000, scheduledTime=1463524600466, launchTime=1463524600466
2016-05-17 22:36:40,467 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524600291_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:40,467 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2]
2016-05-17 22:36:40,467 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [75EB0E8F8C1D426FAE1673C9F1827659/D73FAC7AD9F845A0B50B097DA746432A] each/(1/1) ...lyplatform/0_UNKNOWN_SWAP, VertexName: F9FDDBA2BFD64DF8B8415866318968D2, VertexParallelism: 1, TaskAttemptID:attempt_1463524600291_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E3F76FB2565442C9A69716E2D98205E1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FFF09557E61D45A68269F4A4EACC2037, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:40,467 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524600291_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:40,468 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:40,468 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:40,469 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:40,469 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524600291_0001, with dag id: dag_1463524600291_0001_1
2016-05-17 22:36:40,469 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:40,469 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524600291_0001_00_000001
2016-05-17 22:36:40,469 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524600291_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:40,471 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524600291_0001_00_000001
2016-05-17 22:36:40,471 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:40,473 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524600291_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:40,473 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:40,473 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600291_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524600291_0001_00_000001, launchTime=1463524600473
2016-05-17 22:36:40,474 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524600291_0001_00_000001
2016-05-17 22:36:40,474 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524600291_0001_1_00_000000_0] to container: [container_1463524600291_0001_00_000001]
2016-05-17 22:36:40,474 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524600291_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:40,475 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524600291_0001_1_00_000000_0] started. Is using containerId: [container_1463524600291_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:40,475 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600291_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F9FDDBA2BFD64DF8B8415866318968D2, taskAttemptId=attempt_1463524600291_0001_1_00_000000_0, startTime=1463524600474, containerId=container_1463524600291_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524600291_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:40,475 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524600291_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:40,475 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:40,475 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524600291_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:40,475 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524600291_0001_00_000001 given task: attempt_1463524600291_0001_1_00_000000_0
2016-05-17 22:36:40,475 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524600291_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524600291_0001_1_00_000000_0
2016-05-17 22:36:40 Starting to run new task attempt: attempt_1463524600291_0001_1_00_000000_0
2016-05-17 22:36:40,476 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524600291_0001_1_00_000000_0
2016-05-17 22:36:40,476 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:40,476 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:40,476 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:40,476 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:40,476 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:40,477 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:40,477 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [75EB0E8F8C1D426FAE1673C9F1827659/D73FAC7AD9F845A0B50B097DA746432A] each/(1/1) ...lyplatform/0_UNKNOWN_SWAP, VertexName: F9FDDBA2BFD64DF8B8415866318968D2, VertexParallelism: 1, TaskAttemptID:attempt_1463524600291_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E3F76FB2565442C9A69716E2D98205E1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FFF09557E61D45A68269F4A4EACC2037, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:40,477 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:40,477 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524600291_0001_1_00_000000_0
2016-05-17 22:36:40,477 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:40,478 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:40,478 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524600291_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:40,478 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:40,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E3F76FB2565442C9A69716E2D98205E1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:40,480 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:40,480 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E3F76FB2565442C9A69716E2D98205E1
2016-05-17 22:36:40,480 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: E3F76FB2565442C9A69716E2D98205E1
2016-05-17 22:36:40,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FFF09557E61D45A68269F4A4EACC2037, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:40,481 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:40,481 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:40,481 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FFF09557E61D45A68269F4A4EACC2037
2016-05-17 22:36:40,482 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:40,482 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: FFF09557E61D45A68269F4A4EACC2037
2016-05-17 22:36:40,488 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:40,489 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: E3F76FB2565442C9A69716E2D98205E1
2016-05-17 22:36:40,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E3F76FB2565442C9A69716E2D98205E1
2016-05-17 22:36:40,490 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F9FDDBA2BFD64DF8B8415866318968D2, ordinal: 0
2016-05-17 22:36:40,490 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9FDDBA2BFD64DF8B8415866318968D2, mem on start (mb), free: 281, total: 371, max: 672
2016-05-17 22:36:40,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:40,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:40,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:40,494 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:40,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FFF09557E61D45A68269F4A4EACC2037
2016-05-17 22:36:40,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:40,495 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:40,495 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:40,495 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:40,495 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:40,496 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:40,496 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:40,496 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:40,496 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E3F76FB2565442C9A69716E2D98205E1, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:40,497 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FFF09557E61D45A68269F4A4EACC2037, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:40,497 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:40,497 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E3F76FB2565442C9A69716E2D98205E1
2016-05-17 22:36:40,497 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:40,497 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E3F76FB2565442C9A69716E2D98205E1
2016-05-17 22:36:40,497 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E3F76FB2565442C9A69716E2D98205E1 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:40,498 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:40,498 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:40,498 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:40,499 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524600291_0001_1_00_000000_0
2016-05-17 22:36:40,499 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:40,499 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@13bb724a
2016-05-17 22:36:40,501 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@13bb724a
2016-05-17 22:36:40,501 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:40,501 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:40,502 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:40,502 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: E3F76FB2565442C9A69716E2D98205E1
2016-05-17 22:36:40,502 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_SWAP"], id: FFF09557E61D45A68269F4A4EACC2037
2016-05-17 22:36:40,503 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_SWAP"]
2016-05-17 22:36:40,503 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:40,503 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:40,503 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F9FDDBA2BFD64DF8B8415866318968D2, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:40,506 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:40,506 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:40,506 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524600291_0001_1_00_000000_0
2016-05-17 22:36:40,506 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524600291_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:40,507 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524600291_0001_1_00_000000_0
2016-05-17 22:36:40,507 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9FDDBA2BFD64DF8B8415866318968D2, mem on close (mb), free: 280, total: 371, max: 672
2016-05-17 22:36:40 Completed running task attempt: attempt_1463524600291_0001_1_00_000000_0
2016-05-17 22:36:40,508 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524600291_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:40,509 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600291_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F9FDDBA2BFD64DF8B8415866318968D2, taskAttemptId=attempt_1463524600291_0001_1_00_000000_0, startTime=1463524600474, finishTime=1463524600509, timeTaken=35, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524600503, Process_Duration=4, Process_End_Time=1463524600507, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:40,509 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524600291_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:40,509 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524600291_0001_1_00_000000_0
2016-05-17 22:36:40,509 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:40,510 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600291_0001_1][Event:TASK_FINISHED]: vertexName=F9FDDBA2BFD64DF8B8415866318968D2, taskId=task_1463524600291_0001_1_00_000000, startTime=1463524600474, finishTime=1463524600509, timeTaken=35, status=SUCCEEDED, successfulAttemptID=attempt_1463524600291_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524600503, Process_Duration=4, Process_End_Time=1463524600507, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:40,510 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524600291_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:40,510 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2] : 1
2016-05-17 22:36:40,510 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:40,510 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2]
2016-05-17 22:36:40,511 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:40,511 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600291_0001_1][Event:VERTEX_FINISHED]: vertexName=F9FDDBA2BFD64DF8B8415866318968D2, vertexId=vertex_1463524600291_0001_1_00, initRequestedTime=1463524600454, initedTime=1463524600464, startRequestedTime=1463524600465, startedTime=1463524600465, finishTime=1463524600510, timeTaken=45, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524600503, Process_Duration=4, Process_End_Time=1463524600507, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524600474, firstTasksToStart=[ task_1463524600291_0001_1_00_000000 ], lastTaskFinishTime=1463524600509, lastTasksToFinish=[ task_1463524600291_0001_1_00_000000 ], minTaskDuration=35, maxTaskDuration=35, avgTaskDuration=35.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524600291_0001_1_00_000000 ], longestDurationTasks=[ task_1463524600291_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:40,511 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:40,511 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524600291_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:40,512 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:40,512 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524600291_0001_1
2016-05-17 22:36:40,512 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:40,512 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600291_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524600291_0001_1
2016-05-17 22:36:40,512 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: FFF09557E61D45A68269F4A4EACC2037 for vertex: vertex_1463524600291_0001_1_00 [F9FDDBA2BFD64DF8B8415866318968D2]
2016-05-17 22:36:40,514 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524600503, Process_Duration=4, Process_End_Time=1463524600507, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:40,515 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:40,515 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:40,515 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524600291_0001_00_000001
2016-05-17 22:36:40,521 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524600291_0001_1, queueSize=0
2016-05-17 22:36:40,521 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:40,521 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600291_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524600291_0001_1, startTime=1463524600453, finishTime=1463524600520, timeTaken=67, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524600503, Process_Duration=4, Process_End_Time=1463524600507, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:40,521 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524600291_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:40,521 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524600291_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:40,522 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600291_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524600291_0001_00_000001, stoppedTime=1463524600522, exitStatus=0
2016-05-17 22:36:40,522 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524600291_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:40,522 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:40 Completed Dag: dag_1463524600291_0001_1
2016-05-17 22:36:40,523 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524600291_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:40,523 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524600291_0001_1_post
2016-05-17 22:36:40,523 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:40,524 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:40,524 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524600291_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:36:40,525 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524600291_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:36:40,526 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524600291_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:40,526 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524600291_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:40,526 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524600291_0001_00_000001, Exiting
2016-05-17 22:36:40,526 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524600291_0001_00_000001
2016-05-17 22:36:40,526 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524600291_0001_00_000001
2016-05-17 22:36:40,528 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524600291_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:40,528 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524600291_0001_00_000001 completed successfully
2016-05-17 22:36:40,528 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524600291_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:36:40,529 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/0_UNKNOWN_SWAP, applicationId=application_1463524600291_0001
2016-05-17 22:36:40,530 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:40,531 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:40,531 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:40,531 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:40,531 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:40,531 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:40,531 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:40,532 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:40,532 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:40,532 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:40,532 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:40,532 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:40,532 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:40,537 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/75EB0E8F8C1D426FAE1673C9F1827659/D73FAC7AD9F845A0B50B097DA746432A/.tez/application_1463524600291_0001
2016-05-17 22:36:40,537 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_UNKNOWN_SWAP/_temporary
2016-05-17 22:36:40,537 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.267
2016-05-17 22:36:40,600 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_UNKNOWN_SWAP
2016-05-17 22:36:40,620 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:40,621 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:40,648 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 22:36:40,653 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 22:36:40,653 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:40,654 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:40,654 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:40,667 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:40,667 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_-1
2016-05-17 22:36:40,675 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:40,675 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:40,676 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_-1"]
2016-05-17 22:36:40,676 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:40,676 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:40,676 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:40,676 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...emblyplatform/0_VALUES_-1
2016-05-17 22:36:40,676 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:40,688 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:40,693 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:40,694 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:40,694 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:40,703 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:40,703 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:40,704 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66DD254EDD9F484282BE0BA7CE712FCC/CDF9F191D5424E4DB947F570119B8736/.tez/application_1463524600694_0001 doesn't exist and is created
2016-05-17 22:36:40,720 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:40,720 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:40,721 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66DD254EDD9F484282BE0BA7CE712FCC/CDF9F191D5424E4DB947F570119B8736/.tez/application_1463524600694_0001
2016-05-17 22:36:40,726 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524600694_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:40,736 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:40,737 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:40,739 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:40,739 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:40,740 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66DD254EDD9F484282BE0BA7CE712FCC/CDF9F191D5424E4DB947F570119B8736/.tez/application_1463524600694_0001/localmode-log-dir
2016-05-17 22:36:40,740 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66DD254EDD9F484282BE0BA7CE712FCC/CDF9F191D5424E4DB947F570119B8736/.tez/application_1463524600694_0001/localmode-log-dir/history.txt.appattempt_1463524600694_0001_000000, maxErrors=10
2016-05-17 22:36:40,740 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:40,740 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524600694_0001_000000, appSubmitTime=1463524600725, launchTime=1463524600725
2016-05-17 22:36:40,744 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:40,745 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:40,745 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:40,746 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:40,749 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:55443
2016-05-17 22:36:40,750 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:40,752 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524600694_0001_000000, startTime=1463524600752
2016-05-17 22:36:40,752 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:40,752 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:40,820 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:40,820 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:40,820 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...emblyplatform/0_VALUES_-1, applicationId=application_1463524600694_0001, dagName=[66DD254EDD9F484282BE0BA7CE712FCC/CDF9F191D5424E4DB947F570119B8736] each/(1/1) ...emblyplatform/0_VALUES_-1
2016-05-17 22:36:40,836 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [66DD254EDD9F484282BE0BA7CE712FCC/CDF9F191D5424E4DB947F570119B8736] each/(1/1) ...emblyplatform/0_VALUES_-1
2016-05-17 22:36:40,837 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524600694_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66DD254EDD9F484282BE0BA7CE712FCC/CDF9F191D5424E4DB947F570119B8736/.tez/application_1463524600694_0001/localmode-log-dir/dag_1463524600694_0001_1.dot
2016-05-17 22:36:40,837 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66DD254EDD9F484282BE0BA7CE712FCC/CDF9F191D5424E4DB947F570119B8736/.tez/application_1463524600694_0001/localmode-log-dir/dag_1463524600694_0001_1-tez-dag.pb.txt
2016-05-17 22:36:40,841 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524600694_0001_1
2016-05-17 22:36:40,841 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:40,841 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [66DD254EDD9F484282BE0BA7CE712FCC/CDF9F191D5424E4DB947F570119B8736] each/(1/1) ...emblyplatform/0_VALUES_-1
2016-05-17 22:36:40 Running Dag: dag_1463524600694_0001_1
2016-05-17 22:36:40,852 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:40,853 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600694_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524600694_0001_1, submitTime=1463524600836
2016-05-17 22:36:40,853 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CAD3E5C082DB4D108B47938CFB7E2211
2016-05-17 22:36:40,853 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:40,854 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:40,854 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600694_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524600694_0001_1, initTime=1463524600853
2016-05-17 22:36:40,854 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524600694_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:40,854 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600694_0001_1][Event:DAG_STARTED]: dagID=dag_1463524600694_0001_1, startTime=1463524600854
2016-05-17 22:36:40,854 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:40,854 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:40,854 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524600694_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:40,855 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...emblyplatform/0_VALUES_-1, applicationId=application_1463524600694_0001, dagName=[66DD254EDD9F484282BE0BA7CE712FCC/CDF9F191D5424E4DB947F570119B8736] each/(1/1) ...emblyplatform/0_VALUES_-1
2016-05-17 22:36:40,855 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CAD3E5C082DB4D108B47938CFB7E2211 : {88474DE97C584F62B27EFF1427F02A6B={InputName=88474DE97C584F62B27EFF1427F02A6B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:40,855 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 88474DE97C584F62B27EFF1427F02A6B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:40,855 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211]
2016-05-17 22:36:40,855 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211]
2016-05-17 22:36:40,856 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211]
2016-05-17 22:36:40,856 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211]. Starting root input initializers: 1
2016-05-17 22:36:40,856 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:40,857 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 88474DE97C584F62B27EFF1427F02A6B on vertex vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211]
2016-05-17 22:36:40,858 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:40,858 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 88474DE97C584F62B27EFF1427F02A6B asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:40,859 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:40,859 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:40,860 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:40,861 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 88474DE97C584F62B27EFF1427F02A6B on vertex vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211]
2016-05-17 22:36:40,861 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:40,861 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {88474DE97C584F62B27EFF1427F02A6B=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:40,861 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211] parallelism set to 1
2016-05-17 22:36:40,861 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:40,862 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600694_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524600694_0001_1_00, eventCount=1
2016-05-17 22:36:40,862 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211]
2016-05-17 22:36:40,862 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=01C9D97C83B244D39653A88B7C88CBAD, vertexId=vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:40,862 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=01C9D97C83B244D39653A88B7C88CBAD, vertexId=vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211]
2016-05-17 22:36:40,863 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=01C9D97C83B244D39653A88B7C88CBAD, vertexName=CAD3E5C082DB4D108B47938CFB7E2211, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:40,864 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=01C9D97C83B244D39653A88B7C88CBAD, vertexId=vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211]
2016-05-17 22:36:40,865 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600694_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CAD3E5C082DB4D108B47938CFB7E2211, vertexId=vertex_1463524600694_0001_1_00, initRequestedTime=1463524600855, initedTime=1463524600865, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:40,865 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:40,865 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211] with distanceFromRoot: 0
2016-05-17 22:36:40,865 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:40,865 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CAD3E5C082DB4D108B47938CFB7E2211
2016-05-17 22:36:40,865 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211]
2016-05-17 22:36:40,866 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:40,866 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600694_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524600694_0001_1_00, startRequestedTime=1463524600865, startedTime=1463524600865
2016-05-17 22:36:40,866 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:40,867 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:40,867 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600694_0001_1][Event:TASK_STARTED]: vertexName=CAD3E5C082DB4D108B47938CFB7E2211, taskId=task_1463524600694_0001_1_00_000000, scheduledTime=1463524600866, launchTime=1463524600866
2016-05-17 22:36:40,867 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524600694_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:40,867 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211]
2016-05-17 22:36:40,868 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [66DD254EDD9F484282BE0BA7CE712FCC/CDF9F191D5424E4DB947F570119B8736] each/(1/1) ...emblyplatform/0_VALUES_-1, VertexName: CAD3E5C082DB4D108B47938CFB7E2211, VertexParallelism: 1, TaskAttemptID:attempt_1463524600694_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=88474DE97C584F62B27EFF1427F02A6B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=01C9D97C83B244D39653A88B7C88CBAD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:40,868 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524600694_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:40,868 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:40,869 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:40,869 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:40,869 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:40,870 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524600694_0001, with dag id: dag_1463524600694_0001_1
2016-05-17 22:36:40,870 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524600694_0001_00_000001
2016-05-17 22:36:40,870 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524600694_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:40,871 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524600694_0001_00_000001
2016-05-17 22:36:40,872 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:40,872 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524600694_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:40,873 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:40,873 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600694_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524600694_0001_00_000001, launchTime=1463524600872
2016-05-17 22:36:40,874 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524600694_0001_00_000001
2016-05-17 22:36:40,875 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524600694_0001_1_00_000000_0] to container: [container_1463524600694_0001_00_000001]
2016-05-17 22:36:40,875 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524600694_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:40,876 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524600694_0001_1_00_000000_0] started. Is using containerId: [container_1463524600694_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:40,876 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600694_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CAD3E5C082DB4D108B47938CFB7E2211, taskAttemptId=attempt_1463524600694_0001_1_00_000000_0, startTime=1463524600875, containerId=container_1463524600694_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524600694_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:40,876 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:40,876 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524600694_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:40,876 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524600694_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:40,876 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524600694_0001_00_000001 given task: attempt_1463524600694_0001_1_00_000000_0
2016-05-17 22:36:40,876 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524600694_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524600694_0001_1_00_000000_0
2016-05-17 22:36:40 Starting to run new task attempt: attempt_1463524600694_0001_1_00_000000_0
2016-05-17 22:36:40,877 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524600694_0001_1_00_000000_0
2016-05-17 22:36:40,877 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:40,877 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:40,877 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:40,878 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:40,878 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:40,878 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:40,878 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [66DD254EDD9F484282BE0BA7CE712FCC/CDF9F191D5424E4DB947F570119B8736] each/(1/1) ...emblyplatform/0_VALUES_-1, VertexName: CAD3E5C082DB4D108B47938CFB7E2211, VertexParallelism: 1, TaskAttemptID:attempt_1463524600694_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=88474DE97C584F62B27EFF1427F02A6B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=01C9D97C83B244D39653A88B7C88CBAD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:40,878 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:40,879 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524600694_0001_1_00_000000_0
2016-05-17 22:36:40,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:40,879 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524600694_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:40,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:40,880 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=88474DE97C584F62B27EFF1427F02A6B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:40,880 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=01C9D97C83B244D39653A88B7C88CBAD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:40,880 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:40,880 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:40,881 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 88474DE97C584F62B27EFF1427F02A6B
2016-05-17 22:36:40,881 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 88474DE97C584F62B27EFF1427F02A6B
2016-05-17 22:36:40,881 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:40,881 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 01C9D97C83B244D39653A88B7C88CBAD
2016-05-17 22:36:40,882 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:40,882 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 01C9D97C83B244D39653A88B7C88CBAD
2016-05-17 22:36:40,884 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:40,885 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:40,885 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 88474DE97C584F62B27EFF1427F02A6B
2016-05-17 22:36:40,885 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 88474DE97C584F62B27EFF1427F02A6B
2016-05-17 22:36:40,891 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CAD3E5C082DB4D108B47938CFB7E2211, ordinal: 0
2016-05-17 22:36:40,891 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CAD3E5C082DB4D108B47938CFB7E2211, mem on start (mb), free: 205, total: 371, max: 672
2016-05-17 22:36:40,891 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:40,891 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:40,891 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:40,895 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:40,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 01C9D97C83B244D39653A88B7C88CBAD
2016-05-17 22:36:40,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:40,895 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:40,895 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:40,895 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:40,896 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:40,896 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:40,897 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:40,897 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:40,897 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 88474DE97C584F62B27EFF1427F02A6B, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:40,898 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 01C9D97C83B244D39653A88B7C88CBAD, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:40,898 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:40,898 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 88474DE97C584F62B27EFF1427F02A6B being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:40,898 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 88474DE97C584F62B27EFF1427F02A6B
2016-05-17 22:36:40,898 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:40,898 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 88474DE97C584F62B27EFF1427F02A6B
2016-05-17 22:36:40,898 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:40,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:40,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:40,899 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524600694_0001_1_00_000000_0
2016-05-17 22:36:40,901 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:40,903 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7dac1a45
2016-05-17 22:36:40,903 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:40,903 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 88474DE97C584F62B27EFF1427F02A6B
2016-05-17 22:36:40,904 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_-1"], id: 01C9D97C83B244D39653A88B7C88CBAD
2016-05-17 22:36:40,904 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_-1"]
2016-05-17 22:36:40,904 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:40,904 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:40,905 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7dac1a45
2016-05-17 22:36:40,905 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:40,906 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:40,906 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CAD3E5C082DB4D108B47938CFB7E2211, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:40,907 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:40,911 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:40,914 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:40,915 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:40,915 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524600694_0001_1_00_000000_0
2016-05-17 22:36:40,915 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524600694_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:40,916 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524600694_0001_1_00_000000_0
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:40,921 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:40,921 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600694_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CAD3E5C082DB4D108B47938CFB7E2211, taskAttemptId=attempt_1463524600694_0001_1_00_000000_0, startTime=1463524600875, finishTime=1463524600921, timeTaken=46, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524600906, Process_Duration=10, Process_End_Time=1463524600916, Read_Duration=1, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:36:40,921 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524600694_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:36:40,922 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524600694_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:36:40,922 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:40,922 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600694_0001_1][Event:TASK_FINISHED]: vertexName=CAD3E5C082DB4D108B47938CFB7E2211, taskId=task_1463524600694_0001_1_00_000000, startTime=1463524600875, finishTime=1463524600922, timeTaken=47, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
], counters=Counters: 0
2016-05-17 22:36:40,922 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524600694_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:36:40,922 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:40,922 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211] : 1
2016-05-17 22:36:40,923 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211] because task failed: task_1463524600694_0001_1_00_000000
2016-05-17 22:36:40,923 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:36:40,923 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:36:40,923 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211]
2016-05-17 22:36:40,923 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=01C9D97C83B244D39653A88B7C88CBAD, vertexId=vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211]
2016-05-17 22:36:40,924 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:40,925 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600694_0001_1][Event:VERTEX_FINISHED]: vertexName=CAD3E5C082DB4D108B47938CFB7E2211, vertexId=vertex_1463524600694_0001_1_00, initRequestedTime=1463524600855, initedTime=1463524600865, startRequestedTime=1463524600865, startedTime=1463524600865, finishTime=1463524600924, timeTaken=59, status=FAILED, diagnostics=Task failed, taskId=task_1463524600694_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524600875, firstTasksToStart=[ task_1463524600694_0001_1_00_000000 ], lastTaskFinishTime=1463524600921, lastTasksToFinish=[ task_1463524600694_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:36:40,925 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:36:40,925 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:40,926 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600694_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524600694_0001_00_000001, stoppedTime=1463524600925, exitStatus=0
2016-05-17 22:36:40,926 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524600694_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:40,926 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524600694_0001_1_00_000000_0 failed: true container: container_1463524600694_0001_00_000001 numFailedTAs: 0
2016-05-17 22:36:40,926 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:36:40,926 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:36:40,926 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524600694_0001_1
2016-05-17 22:36:40,926 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 01C9D97C83B244D39653A88B7C88CBAD for vertex: vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211]
2016-05-17 22:36:40,927 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524600694_0001_1, queueSize=0
2016-05-17 22:36:40,927 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:40,928 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524600694_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524600694_0001_1, startTime=1463524600854, finishTime=1463524600927, timeTaken=73, status=FAILED, diagnostics=Vertex failed, vertexName=CAD3E5C082DB4D108B47938CFB7E2211, vertexId=vertex_1463524600694_0001_1_00, diagnostics=[Task failed, taskId=task_1463524600694_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524600694_0001_1_00 [CAD3E5C082DB4D108B47938CFB7E2211] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:36:40,928 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524600694_0001_1 finished with state: FAILED
2016-05-17 22:36:40,928 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524600694_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:40,928 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524600694_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:36:40 Completed Dag: dag_1463524600694_0001_1
2016-05-17 22:36:40,929 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524600694_0001_1, dagState=FAILED
2016-05-17 22:36:40,929 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524600694_0001_1_post
2016-05-17 22:36:40,929 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:40,929 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:40,929 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524600694_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:36:40,927 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 19 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524600906, Process_Duration=10, Process_End_Time=1463524600916, Read_Duration=1, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:36:40,929 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CAD3E5C082DB4D108B47938CFB7E2211, mem on close (mb), free: 202, total: 371, max: 672
2016-05-17 22:36:40,929 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=CAD3E5C082DB4D108B47938CFB7E2211, index=0
2016-05-17 22:36:40,930 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=CAD3E5C082DB4D108B47938CFB7E2211, sourceVertex=88474DE97C584F62B27EFF1427F02A6B
2016-05-17 22:36:40,930 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=CAD3E5C082DB4D108B47938CFB7E2211, sourceVertex=01C9D97C83B244D39653A88B7C88CBAD
2016-05-17 22:36:40,930 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524600694_0001_00_000001
2016-05-17 22:36:40,930 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524600694_0001_00_000001
2016-05-17 22:36:40,934 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524600694_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:36:40,934 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524600694_0001_00_000001 completed but with errors
2016-05-17 22:36:40,934 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524600694_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:36:40,934 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524600694_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:36:40,934 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524600694_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:36:40,934 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...emblyplatform/0_VALUES_-1, applicationId=application_1463524600694_0001
2016-05-17 22:36:40,936 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:40,936 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:40,936 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:40,936 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:40,936 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:40,937 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:40,937 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:40,937 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:40,937 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:40,938 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:40,938 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:40,938 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:40,938 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:40,942 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66DD254EDD9F484282BE0BA7CE712FCC/CDF9F191D5424E4DB947F570119B8736/.tez/application_1463524600694_0001
2016-05-17 22:36:40,942 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping all jobs
2016-05-17 22:36:40,942 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping: (1/1) ...emblyplatform/0_VALUES_-1
2016-05-17 22:36:40,942 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopped all jobs
2016-05-17 22:36:40,943 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_-1/_temporary
2016-05-17 22:36:40,943 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.268
2016-05-17 22:36:40,943 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_VALUES_-1
2016-05-17 22:36:40,964 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:40,964 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:40,988 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.002
2016-05-17 22:36:40,993 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.007
2016-05-17 22:36:40,993 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:40,993 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:40,994 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_VALUES_'field'
2016-05-17 22:36:41,014 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:41,014 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:41,045 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 22:36:41,049 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 22:36:41,049 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:41,049 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:41,050 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:41,062 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:41,063 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_RESULTS
2016-05-17 22:36:41,071 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:41,071 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:41,071 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_RESULTS"]
2016-05-17 22:36:41,072 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:41,072 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:41,072 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:41,074 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...platform/0_VALUES_RESULTS
2016-05-17 22:36:41,074 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:41,085 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:41,089 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:41,090 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:41,090 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:41,100 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/52564B29F45D461FB8BB3E45C1E4A04B/E293ADFDCAC64F8C862D0E8C5DC86A9F/.tez/application_1463524601090_0001 doesn't exist and is created
2016-05-17 22:36:41,118 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:41,118 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:41,118 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/52564B29F45D461FB8BB3E45C1E4A04B/E293ADFDCAC64F8C862D0E8C5DC86A9F/.tez/application_1463524601090_0001
2016-05-17 22:36:41,121 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524601090_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:41,131 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:41,132 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:41,133 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:41,133 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:41,134 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/52564B29F45D461FB8BB3E45C1E4A04B/E293ADFDCAC64F8C862D0E8C5DC86A9F/.tez/application_1463524601090_0001/localmode-log-dir
2016-05-17 22:36:41,134 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/52564B29F45D461FB8BB3E45C1E4A04B/E293ADFDCAC64F8C862D0E8C5DC86A9F/.tez/application_1463524601090_0001/localmode-log-dir/history.txt.appattempt_1463524601090_0001_000000, maxErrors=10
2016-05-17 22:36:41,134 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:41,134 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524601090_0001_000000, appSubmitTime=1463524601121, launchTime=1463524601121
2016-05-17 22:36:41,137 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:41,138 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:41,139 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:41,139 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:41,139 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:41,140 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:41,145 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:45412
2016-05-17 22:36:41,146 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:41,148 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524601090_0001_000000, startTime=1463524601148
2016-05-17 22:36:41,148 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:41,148 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:41,219 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:41,219 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:41,219 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/0_VALUES_RESULTS, applicationId=application_1463524601090_0001, dagName=[52564B29F45D461FB8BB3E45C1E4A04B/E293ADFDCAC64F8C862D0E8C5DC86A9F] each/(1/1) ...platform/0_VALUES_RESULTS
2016-05-17 22:36:41,233 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [52564B29F45D461FB8BB3E45C1E4A04B/E293ADFDCAC64F8C862D0E8C5DC86A9F] each/(1/1) ...platform/0_VALUES_RESULTS
2016-05-17 22:36:41,235 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524601090_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/52564B29F45D461FB8BB3E45C1E4A04B/E293ADFDCAC64F8C862D0E8C5DC86A9F/.tez/application_1463524601090_0001/localmode-log-dir/dag_1463524601090_0001_1.dot
2016-05-17 22:36:41,235 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/52564B29F45D461FB8BB3E45C1E4A04B/E293ADFDCAC64F8C862D0E8C5DC86A9F/.tez/application_1463524601090_0001/localmode-log-dir/dag_1463524601090_0001_1-tez-dag.pb.txt
2016-05-17 22:36:41,239 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524601090_0001_1
2016-05-17 22:36:41,239 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:41,239 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [52564B29F45D461FB8BB3E45C1E4A04B/E293ADFDCAC64F8C862D0E8C5DC86A9F] each/(1/1) ...platform/0_VALUES_RESULTS
2016-05-17 22:36:41 Running Dag: dag_1463524601090_0001_1
2016-05-17 22:36:41,252 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601090_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524601090_0001_1, submitTime=1463524601234
2016-05-17 22:36:41,252 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:41,252 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BCA9F43903DF4DB987351D0C79BBB3CE
2016-05-17 22:36:41,253 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:41,253 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601090_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524601090_0001_1, initTime=1463524601252
2016-05-17 22:36:41,253 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:41,253 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524601090_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:41,254 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601090_0001_1][Event:DAG_STARTED]: dagID=dag_1463524601090_0001_1, startTime=1463524601254
2016-05-17 22:36:41,254 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:41,254 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:41,254 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524601090_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:41,254 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BCA9F43903DF4DB987351D0C79BBB3CE : {1CBEDF0E89E046CD849CBF4A288C5671={InputName=1CBEDF0E89E046CD849CBF4A288C5671}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:41,254 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 1CBEDF0E89E046CD849CBF4A288C5671, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:41,254 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE]
2016-05-17 22:36:41,255 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE]
2016-05-17 22:36:41,255 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE]
2016-05-17 22:36:41,255 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE]. Starting root input initializers: 1
2016-05-17 22:36:41,255 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/0_VALUES_RESULTS, applicationId=application_1463524601090_0001, dagName=[52564B29F45D461FB8BB3E45C1E4A04B/E293ADFDCAC64F8C862D0E8C5DC86A9F] each/(1/1) ...platform/0_VALUES_RESULTS
2016-05-17 22:36:41,255 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:41,256 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 1CBEDF0E89E046CD849CBF4A288C5671 on vertex vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE]
2016-05-17 22:36:41,257 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:41,258 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 1CBEDF0E89E046CD849CBF4A288C5671 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:41,258 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:41,258 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:41,260 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:41,260 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 1CBEDF0E89E046CD849CBF4A288C5671 on vertex vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE]
2016-05-17 22:36:41,260 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:41,261 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {1CBEDF0E89E046CD849CBF4A288C5671=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:41,261 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE] parallelism set to 1
2016-05-17 22:36:41,261 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:41,261 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601090_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524601090_0001_1_00, eventCount=1
2016-05-17 22:36:41,261 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE]
2016-05-17 22:36:41,261 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=7A18DA5495524E45BBB42687C40C6643, vertexId=vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:41,262 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=7A18DA5495524E45BBB42687C40C6643, vertexId=vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE]
2016-05-17 22:36:41,264 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=7A18DA5495524E45BBB42687C40C6643, vertexName=BCA9F43903DF4DB987351D0C79BBB3CE, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:41,264 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=7A18DA5495524E45BBB42687C40C6643, vertexId=vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE]
2016-05-17 22:36:41,265 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:41,265 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601090_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BCA9F43903DF4DB987351D0C79BBB3CE, vertexId=vertex_1463524601090_0001_1_00, initRequestedTime=1463524601254, initedTime=1463524601265, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:41,265 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE] with distanceFromRoot: 0
2016-05-17 22:36:41,265 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:41,266 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BCA9F43903DF4DB987351D0C79BBB3CE
2016-05-17 22:36:41,266 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE]
2016-05-17 22:36:41,266 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:41,266 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601090_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524601090_0001_1_00, startRequestedTime=1463524601266, startedTime=1463524601266
2016-05-17 22:36:41,266 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:41,266 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:41,267 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601090_0001_1][Event:TASK_STARTED]: vertexName=BCA9F43903DF4DB987351D0C79BBB3CE, taskId=task_1463524601090_0001_1_00_000000, scheduledTime=1463524601266, launchTime=1463524601266
2016-05-17 22:36:41,267 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524601090_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:41,267 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE]
2016-05-17 22:36:41,267 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [52564B29F45D461FB8BB3E45C1E4A04B/E293ADFDCAC64F8C862D0E8C5DC86A9F] each/(1/1) ...platform/0_VALUES_RESULTS, VertexName: BCA9F43903DF4DB987351D0C79BBB3CE, VertexParallelism: 1, TaskAttemptID:attempt_1463524601090_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=1CBEDF0E89E046CD849CBF4A288C5671, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=7A18DA5495524E45BBB42687C40C6643, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:41,267 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524601090_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:41,268 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:41,268 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:41,269 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:41,269 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:41,269 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524601090_0001_00_000001
2016-05-17 22:36:41,269 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524601090_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:41,271 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524601090_0001_00_000001
2016-05-17 22:36:41,271 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:41,272 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524601090_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:41,272 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:41,273 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601090_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524601090_0001_00_000001, launchTime=1463524601272
2016-05-17 22:36:41,273 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524601090_0001_00_000001
2016-05-17 22:36:41,274 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524601090_0001_1_00_000000_0] to container: [container_1463524601090_0001_00_000001]
2016-05-17 22:36:41,274 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524601090_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:41,275 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524601090_0001_1_00_000000_0] started. Is using containerId: [container_1463524601090_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:41,275 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601090_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BCA9F43903DF4DB987351D0C79BBB3CE, taskAttemptId=attempt_1463524601090_0001_1_00_000000_0, startTime=1463524601275, containerId=container_1463524601090_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524601090_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:41,275 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524601090_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:41,275 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524601090_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:41,276 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:41,276 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524601090_0001_00_000001 given task: attempt_1463524601090_0001_1_00_000000_0
2016-05-17 22:36:41,276 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524601090_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524601090_0001_1_00_000000_0
2016-05-17 22:36:41 Starting to run new task attempt: attempt_1463524601090_0001_1_00_000000_0
2016-05-17 22:36:41,277 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524601090_0001_1_00_000000_0
2016-05-17 22:36:41,277 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:41,277 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:41,277 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:41,277 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:41,277 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:41,278 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:41,278 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [52564B29F45D461FB8BB3E45C1E4A04B/E293ADFDCAC64F8C862D0E8C5DC86A9F] each/(1/1) ...platform/0_VALUES_RESULTS, VertexName: BCA9F43903DF4DB987351D0C79BBB3CE, VertexParallelism: 1, TaskAttemptID:attempt_1463524601090_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=1CBEDF0E89E046CD849CBF4A288C5671, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=7A18DA5495524E45BBB42687C40C6643, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:41,278 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524601090_0001, with dag id: dag_1463524601090_0001_1
2016-05-17 22:36:41,278 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:41,279 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524601090_0001_1_00_000000_0
2016-05-17 22:36:41,279 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:41,279 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:41,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=1CBEDF0E89E046CD849CBF4A288C5671, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:41,283 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:41,283 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524601090_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:41,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=7A18DA5495524E45BBB42687C40C6643, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:41,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:41,287 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:41,287 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:41,288 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 7A18DA5495524E45BBB42687C40C6643
2016-05-17 22:36:41,288 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:41,288 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 7A18DA5495524E45BBB42687C40C6643
2016-05-17 22:36:41,294 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 1CBEDF0E89E046CD849CBF4A288C5671
2016-05-17 22:36:41,294 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 1CBEDF0E89E046CD849CBF4A288C5671
2016-05-17 22:36:41,296 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:41,296 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 1CBEDF0E89E046CD849CBF4A288C5671
2016-05-17 22:36:41,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 1CBEDF0E89E046CD849CBF4A288C5671
2016-05-17 22:36:41,302 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:41,302 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 7A18DA5495524E45BBB42687C40C6643
2016-05-17 22:36:41,303 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BCA9F43903DF4DB987351D0C79BBB3CE, ordinal: 0
2016-05-17 22:36:41,303 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BCA9F43903DF4DB987351D0C79BBB3CE, mem on start (mb), free: 128, total: 371, max: 672
2016-05-17 22:36:41,303 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:41,303 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:41,304 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:41,304 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:41,304 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:41,304 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:41,304 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:41,305 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:41,305 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:41,305 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:41,305 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:41,305 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 7A18DA5495524E45BBB42687C40C6643, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:41,306 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 1CBEDF0E89E046CD849CBF4A288C5671, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:41,306 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:41,306 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 1CBEDF0E89E046CD849CBF4A288C5671 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:41,306 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 1CBEDF0E89E046CD849CBF4A288C5671
2016-05-17 22:36:41,306 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:41,307 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:41,306 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:41,307 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 1CBEDF0E89E046CD849CBF4A288C5671
2016-05-17 22:36:41,307 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:41,308 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524601090_0001_1_00_000000_0
2016-05-17 22:36:41,308 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:41,308 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6d866a6f
2016-05-17 22:36:41,309 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6d866a6f
2016-05-17 22:36:41,309 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:41,309 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:41,311 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:41,312 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 1CBEDF0E89E046CD849CBF4A288C5671
2016-05-17 22:36:41,312 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_RESULTS"], id: 7A18DA5495524E45BBB42687C40C6643
2016-05-17 22:36:41,312 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_RESULTS"]
2016-05-17 22:36:41,313 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:41,313 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:41,313 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BCA9F43903DF4DB987351D0C79BBB3CE, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:41,314 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:41,314 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:41,316 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:41,316 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:41,316 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524601090_0001_1_00_000000_0
2016-05-17 22:36:41,316 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524601090_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:41,317 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524601090_0001_1_00_000000_0
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:41,319 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601090_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BCA9F43903DF4DB987351D0C79BBB3CE, taskAttemptId=attempt_1463524601090_0001_1_00_000000_0, startTime=1463524601275, finishTime=1463524601318, timeTaken=43, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524601313, Process_Duration=4, Process_End_Time=1463524601317, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:36:41,319 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524601090_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:36:41,319 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524601090_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:36:41,319 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601090_0001_1][Event:TASK_FINISHED]: vertexName=BCA9F43903DF4DB987351D0C79BBB3CE, taskId=task_1463524601090_0001_1_00_000000, startTime=1463524601275, finishTime=1463524601319, timeTaken=44, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
], counters=Counters: 0
2016-05-17 22:36:41,319 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524601090_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:36:41,320 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE] : 1
2016-05-17 22:36:41,320 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE] because task failed: task_1463524601090_0001_1_00_000000
2016-05-17 22:36:41,320 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:36:41,320 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:36:41,320 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE]
2016-05-17 22:36:41,320 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=7A18DA5495524E45BBB42687C40C6643, vertexId=vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE]
2016-05-17 22:36:41,321 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:41,322 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:41,322 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:41,323 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601090_0001_1][Event:VERTEX_FINISHED]: vertexName=BCA9F43903DF4DB987351D0C79BBB3CE, vertexId=vertex_1463524601090_0001_1_00, initRequestedTime=1463524601254, initedTime=1463524601265, startRequestedTime=1463524601266, startedTime=1463524601266, finishTime=1463524601321, timeTaken=55, status=FAILED, diagnostics=Task failed, taskId=task_1463524601090_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524601275, firstTasksToStart=[ task_1463524601090_0001_1_00_000000 ], lastTaskFinishTime=1463524601318, lastTasksToFinish=[ task_1463524601090_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:36:41,323 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:36:41,323 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:36:41,323 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:36:41,323 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524601090_0001_1
2016-05-17 22:36:41,323 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 7A18DA5495524E45BBB42687C40C6643 for vertex: vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE]
2016-05-17 22:36:41,324 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:41,324 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 19 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524601313, Process_Duration=4, Process_End_Time=1463524601317, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:36:41,324 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524601090_0001_1, queueSize=0
2016-05-17 22:36:41,325 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:41,325 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601090_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524601090_0001_1, startTime=1463524601254, finishTime=1463524601324, timeTaken=70, status=FAILED, diagnostics=Vertex failed, vertexName=BCA9F43903DF4DB987351D0C79BBB3CE, vertexId=vertex_1463524601090_0001_1_00, diagnostics=[Task failed, taskId=task_1463524601090_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524601090_0001_1_00 [BCA9F43903DF4DB987351D0C79BBB3CE] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:36:41,325 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524601090_0001_1 finished with state: FAILED
2016-05-17 22:36:41,325 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BCA9F43903DF4DB987351D0C79BBB3CE, mem on close (mb), free: 125, total: 371, max: 672
2016-05-17 22:36:41,326 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524601090_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:41,326 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:41,326 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=BCA9F43903DF4DB987351D0C79BBB3CE, index=0
2016-05-17 22:36:41,326 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601090_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524601090_0001_00_000001, stoppedTime=1463524601326, exitStatus=0
2016-05-17 22:36:41,326 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=BCA9F43903DF4DB987351D0C79BBB3CE, sourceVertex=1CBEDF0E89E046CD849CBF4A288C5671
2016-05-17 22:36:41,327 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524601090_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:41,327 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524601090_0001_1_00_000000_0 failed: true container: container_1463524601090_0001_00_000001 numFailedTAs: 0
2016-05-17 22:36:41,327 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=BCA9F43903DF4DB987351D0C79BBB3CE, sourceVertex=7A18DA5495524E45BBB42687C40C6643
2016-05-17 22:36:41,328 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524601090_0001_00_000001
2016-05-17 22:36:41,328 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524601090_0001_00_000001
2016-05-17 22:36:41 Completed Dag: dag_1463524601090_0001_1
2016-05-17 22:36:41,329 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524601090_0001_1, dagState=FAILED
2016-05-17 22:36:41,330 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524601090_0001_1_post
2016-05-17 22:36:41,330 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:41,330 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:41,330 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524601090_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:36:41,330 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524601090_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:36:41,333 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524601090_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:36:41,333 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524601090_0001_00_000001 completed but with errors
2016-05-17 22:36:41,333 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524601090_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:36:41,333 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524601090_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:36:41,333 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524601090_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:36:41,337 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/0_VALUES_RESULTS, applicationId=application_1463524601090_0001
2016-05-17 22:36:41,339 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:41,339 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:41,339 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:41,339 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:41,339 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:41,340 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:41,340 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:41,340 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:41,341 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:41,341 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:41,342 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:41,342 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:41,342 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:41,347 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/52564B29F45D461FB8BB3E45C1E4A04B/E293ADFDCAC64F8C862D0E8C5DC86A9F/.tez/application_1463524601090_0001
2016-05-17 22:36:41,347 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping all jobs
2016-05-17 22:36:41,347 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping: (1/1) ...platform/0_VALUES_RESULTS
2016-05-17 22:36:41,348 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopped all jobs
2016-05-17 22:36:41,348 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_RESULTS/_temporary
2016-05-17 22:36:41,348 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.277
2016-05-17 22:36:41,348 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_VALUES_RESULTS
2016-05-17 22:36:41,380 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:41,381 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:41,418 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 22:36:41,436 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-05-17 22:36:41,436 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:41,436 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:41,437 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:41,452 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:41,452 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_ALL
2016-05-17 22:36:41,462 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:41,463 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:41,463 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_ALL"]
2016-05-17 22:36:41,463 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:41,463 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:41,463 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:41,464 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...mblyplatform/0_VALUES_ALL
2016-05-17 22:36:41,464 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:41,475 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:41,480 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:41,480 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:41,480 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:41,490 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D661A8B9D5774EC9A66C94456402C76A/F5E33BD7A40C45D5AEECAE1F2B8B588E/.tez/application_1463524601480_0001 doesn't exist and is created
2016-05-17 22:36:41,506 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:41,506 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:41,507 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D661A8B9D5774EC9A66C94456402C76A/F5E33BD7A40C45D5AEECAE1F2B8B588E/.tez/application_1463524601480_0001
2016-05-17 22:36:41,509 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524601480_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:41,520 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:41,520 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:41,522 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:41,522 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:41,522 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D661A8B9D5774EC9A66C94456402C76A/F5E33BD7A40C45D5AEECAE1F2B8B588E/.tez/application_1463524601480_0001/localmode-log-dir
2016-05-17 22:36:41,522 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D661A8B9D5774EC9A66C94456402C76A/F5E33BD7A40C45D5AEECAE1F2B8B588E/.tez/application_1463524601480_0001/localmode-log-dir/history.txt.appattempt_1463524601480_0001_000000, maxErrors=10
2016-05-17 22:36:41,522 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:41,522 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524601480_0001_000000, appSubmitTime=1463524601509, launchTime=1463524601509
2016-05-17 22:36:41,527 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:41,528 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:41,528 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:43025
2016-05-17 22:36:41,531 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:41,531 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:41,531 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:41,533 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524601480_0001_000000, startTime=1463524601533
2016-05-17 22:36:41,533 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:41,533 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:41,569 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:41,569 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:41,607 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:41,607 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:41,607 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...mblyplatform/0_VALUES_ALL, applicationId=application_1463524601480_0001, dagName=[D661A8B9D5774EC9A66C94456402C76A/F5E33BD7A40C45D5AEECAE1F2B8B588E] each/(1/1) ...mblyplatform/0_VALUES_ALL
2016-05-17 22:36:41,622 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [D661A8B9D5774EC9A66C94456402C76A/F5E33BD7A40C45D5AEECAE1F2B8B588E] each/(1/1) ...mblyplatform/0_VALUES_ALL
2016-05-17 22:36:41,624 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524601480_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D661A8B9D5774EC9A66C94456402C76A/F5E33BD7A40C45D5AEECAE1F2B8B588E/.tez/application_1463524601480_0001/localmode-log-dir/dag_1463524601480_0001_1.dot
2016-05-17 22:36:41,625 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D661A8B9D5774EC9A66C94456402C76A/F5E33BD7A40C45D5AEECAE1F2B8B588E/.tez/application_1463524601480_0001/localmode-log-dir/dag_1463524601480_0001_1-tez-dag.pb.txt
2016-05-17 22:36:41,630 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524601480_0001_1
2016-05-17 22:36:41,630 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:41,631 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [D661A8B9D5774EC9A66C94456402C76A/F5E33BD7A40C45D5AEECAE1F2B8B588E] each/(1/1) ...mblyplatform/0_VALUES_ALL
2016-05-17 22:36:41 Running Dag: dag_1463524601480_0001_1
2016-05-17 22:36:41,643 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601480_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524601480_0001_1, submitTime=1463524601623
2016-05-17 22:36:41,643 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:41,644 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CFEA911CACD2465C9FB160575547F456
2016-05-17 22:36:41,644 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:41,644 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601480_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524601480_0001_1, initTime=1463524601643
2016-05-17 22:36:41,644 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:41,644 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524601480_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:41,644 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601480_0001_1][Event:DAG_STARTED]: dagID=dag_1463524601480_0001_1, startTime=1463524601644
2016-05-17 22:36:41,645 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:41,645 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524601480_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:41,645 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...mblyplatform/0_VALUES_ALL, applicationId=application_1463524601480_0001, dagName=[D661A8B9D5774EC9A66C94456402C76A/F5E33BD7A40C45D5AEECAE1F2B8B588E] each/(1/1) ...mblyplatform/0_VALUES_ALL
2016-05-17 22:36:41,645 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:41,645 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CFEA911CACD2465C9FB160575547F456 : {73CBA8433CD6411AB1655CA54808855C={InputName=73CBA8433CD6411AB1655CA54808855C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:41,645 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 73CBA8433CD6411AB1655CA54808855C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:41,645 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456]
2016-05-17 22:36:41,645 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456]
2016-05-17 22:36:41,645 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456]
2016-05-17 22:36:41,646 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456]. Starting root input initializers: 1
2016-05-17 22:36:41,646 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:41,647 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 73CBA8433CD6411AB1655CA54808855C on vertex vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456]
2016-05-17 22:36:41,648 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:41,649 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 73CBA8433CD6411AB1655CA54808855C asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:41,649 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:41,649 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:41,650 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:41,651 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 73CBA8433CD6411AB1655CA54808855C on vertex vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456]
2016-05-17 22:36:41,651 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:41,651 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {73CBA8433CD6411AB1655CA54808855C=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:41,651 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456] parallelism set to 1
2016-05-17 22:36:41,651 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:41,652 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601480_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524601480_0001_1_00, eventCount=1
2016-05-17 22:36:41,652 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456]
2016-05-17 22:36:41,652 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=FEF5163C5F464B95A38C141510C20C51, vertexId=vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:41,652 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=FEF5163C5F464B95A38C141510C20C51, vertexId=vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456]
2016-05-17 22:36:41,654 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=FEF5163C5F464B95A38C141510C20C51, vertexName=CFEA911CACD2465C9FB160575547F456, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:41,654 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=FEF5163C5F464B95A38C141510C20C51, vertexId=vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456]
2016-05-17 22:36:41,655 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:41,655 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601480_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CFEA911CACD2465C9FB160575547F456, vertexId=vertex_1463524601480_0001_1_00, initRequestedTime=1463524601645, initedTime=1463524601654, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:41,655 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456] with distanceFromRoot: 0
2016-05-17 22:36:41,655 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:41,655 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CFEA911CACD2465C9FB160575547F456
2016-05-17 22:36:41,655 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456]
2016-05-17 22:36:41,655 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:41,656 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601480_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524601480_0001_1_00, startRequestedTime=1463524601655, startedTime=1463524601655
2016-05-17 22:36:41,656 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:41,656 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:41,657 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601480_0001_1][Event:TASK_STARTED]: vertexName=CFEA911CACD2465C9FB160575547F456, taskId=task_1463524601480_0001_1_00_000000, scheduledTime=1463524601656, launchTime=1463524601656
2016-05-17 22:36:41,657 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524601480_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:41,657 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456]
2016-05-17 22:36:41,657 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D661A8B9D5774EC9A66C94456402C76A/F5E33BD7A40C45D5AEECAE1F2B8B588E] each/(1/1) ...mblyplatform/0_VALUES_ALL, VertexName: CFEA911CACD2465C9FB160575547F456, VertexParallelism: 1, TaskAttemptID:attempt_1463524601480_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=73CBA8433CD6411AB1655CA54808855C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FEF5163C5F464B95A38C141510C20C51, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:41,657 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524601480_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:41,658 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:41,658 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:41,659 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:41,659 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:41,659 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524601480_0001_00_000001
2016-05-17 22:36:41,659 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524601480_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:41,661 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524601480_0001_00_000001
2016-05-17 22:36:41,661 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:41,662 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:41,662 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524601480_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:41,662 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601480_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524601480_0001_00_000001, launchTime=1463524601662
2016-05-17 22:36:41,663 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524601480_0001_00_000001
2016-05-17 22:36:41,663 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524601480_0001_1_00_000000_0] to container: [container_1463524601480_0001_00_000001]
2016-05-17 22:36:41,663 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524601480_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:41,664 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524601480_0001_1_00_000000_0] started. Is using containerId: [container_1463524601480_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:41,664 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601480_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CFEA911CACD2465C9FB160575547F456, taskAttemptId=attempt_1463524601480_0001_1_00_000000_0, startTime=1463524601664, containerId=container_1463524601480_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524601480_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:41,664 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524601480_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:41,664 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524601480_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:41,664 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:41,665 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524601480_0001_00_000001 given task: attempt_1463524601480_0001_1_00_000000_0
2016-05-17 22:36:41,665 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524601480_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524601480_0001_1_00_000000_0
2016-05-17 22:36:41 Starting to run new task attempt: attempt_1463524601480_0001_1_00_000000_0
2016-05-17 22:36:41,665 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524601480_0001_1_00_000000_0
2016-05-17 22:36:41,665 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:41,666 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:41,666 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:41,666 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:41,666 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:41,666 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:41,667 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D661A8B9D5774EC9A66C94456402C76A/F5E33BD7A40C45D5AEECAE1F2B8B588E] each/(1/1) ...mblyplatform/0_VALUES_ALL, VertexName: CFEA911CACD2465C9FB160575547F456, VertexParallelism: 1, TaskAttemptID:attempt_1463524601480_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=73CBA8433CD6411AB1655CA54808855C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FEF5163C5F464B95A38C141510C20C51, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:41,667 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524601480_0001, with dag id: dag_1463524601480_0001_1
2016-05-17 22:36:41,667 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:41,668 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524601480_0001_1_00_000000_0
2016-05-17 22:36:41,668 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:41,668 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:41,669 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:41,669 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=73CBA8433CD6411AB1655CA54808855C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:41,669 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:41,669 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 73CBA8433CD6411AB1655CA54808855C
2016-05-17 22:36:41,669 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 73CBA8433CD6411AB1655CA54808855C
2016-05-17 22:36:41,671 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:41,671 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 73CBA8433CD6411AB1655CA54808855C
2016-05-17 22:36:41,671 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 73CBA8433CD6411AB1655CA54808855C
2016-05-17 22:36:41,671 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:41,671 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FEF5163C5F464B95A38C141510C20C51, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:41,672 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:41,672 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FEF5163C5F464B95A38C141510C20C51
2016-05-17 22:36:41,672 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:41,672 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: FEF5163C5F464B95A38C141510C20C51
2016-05-17 22:36:41,675 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524601480_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:41,679 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CFEA911CACD2465C9FB160575547F456, ordinal: 0
2016-05-17 22:36:41,679 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CFEA911CACD2465C9FB160575547F456, mem on start (mb), free: 264, total: 364, max: 672
2016-05-17 22:36:41,679 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:41,679 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:41,679 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:41,686 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:41,686 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FEF5163C5F464B95A38C141510C20C51
2016-05-17 22:36:41,686 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:41,687 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:41,687 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:41,687 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:41,688 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:41,688 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:41,689 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:41,690 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:41,690 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 73CBA8433CD6411AB1655CA54808855C, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:41,690 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FEF5163C5F464B95A38C141510C20C51, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:41,690 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:41,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 73CBA8433CD6411AB1655CA54808855C being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:41,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 73CBA8433CD6411AB1655CA54808855C
2016-05-17 22:36:41,691 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:41,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 73CBA8433CD6411AB1655CA54808855C
2016-05-17 22:36:41,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:41,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:41,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:41,692 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524601480_0001_1_00_000000_0
2016-05-17 22:36:41,692 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:41,693 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2628245c
2016-05-17 22:36:41,693 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2628245c
2016-05-17 22:36:41,694 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:41,694 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:41,696 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:41,696 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 73CBA8433CD6411AB1655CA54808855C
2016-05-17 22:36:41,697 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_ALL"], id: FEF5163C5F464B95A38C141510C20C51
2016-05-17 22:36:41,697 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_ALL"]
2016-05-17 22:36:41,697 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:41,697 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:41,698 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CFEA911CACD2465C9FB160575547F456, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:41,698 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:41,699 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:41,700 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:41,700 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:41,701 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524601480_0001_1_00_000000_0
2016-05-17 22:36:41,701 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524601480_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:41,702 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524601480_0001_1_00_000000_0
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:41,704 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524601698, Process_Duration=4, Process_End_Time=1463524601702, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:36:41,704 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:41,704 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601480_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CFEA911CACD2465C9FB160575547F456, taskAttemptId=attempt_1463524601480_0001_1_00_000000_0, startTime=1463524601664, finishTime=1463524601704, timeTaken=40, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524601698, Process_Duration=4, Process_End_Time=1463524601702, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:36:41,704 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524601480_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:36:41,704 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524601480_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:36:41,705 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601480_0001_1][Event:TASK_FINISHED]: vertexName=CFEA911CACD2465C9FB160575547F456, taskId=task_1463524601480_0001_1_00_000000, startTime=1463524601664, finishTime=1463524601705, timeTaken=41, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
], counters=Counters: 0
2016-05-17 22:36:41,705 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:41,705 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524601480_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:36:41,705 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:41,705 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456] : 1
2016-05-17 22:36:41,707 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456] because task failed: task_1463524601480_0001_1_00_000000
2016-05-17 22:36:41,707 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:36:41,707 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:36:41,707 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456]
2016-05-17 22:36:41,708 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=FEF5163C5F464B95A38C141510C20C51, vertexId=vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456]
2016-05-17 22:36:41,707 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CFEA911CACD2465C9FB160575547F456, mem on close (mb), free: 262, total: 364, max: 672
2016-05-17 22:36:41,708 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=CFEA911CACD2465C9FB160575547F456, index=0
2016-05-17 22:36:41,709 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601480_0001_1][Event:VERTEX_FINISHED]: vertexName=CFEA911CACD2465C9FB160575547F456, vertexId=vertex_1463524601480_0001_1_00, initRequestedTime=1463524601645, initedTime=1463524601654, startRequestedTime=1463524601655, startedTime=1463524601655, finishTime=1463524601708, timeTaken=53, status=FAILED, diagnostics=Task failed, taskId=task_1463524601480_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524601664, firstTasksToStart=[ task_1463524601480_0001_1_00_000000 ], lastTaskFinishTime=1463524601704, lastTasksToFinish=[ task_1463524601480_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:36:41,709 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:36:41,709 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601480_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524601480_0001_00_000001, stoppedTime=1463524601709, exitStatus=0
2016-05-17 22:36:41,709 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524601480_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:41,709 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524601480_0001_1_00_000000_0 failed: true container: container_1463524601480_0001_00_000001 numFailedTAs: 0
2016-05-17 22:36:41,710 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:36:41,710 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:36:41,710 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524601480_0001_1
2016-05-17 22:36:41,710 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: FEF5163C5F464B95A38C141510C20C51 for vertex: vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456]
2016-05-17 22:36:41,710 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524601480_0001_1, queueSize=0
2016-05-17 22:36:41,711 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601480_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524601480_0001_1, startTime=1463524601644, finishTime=1463524601710, timeTaken=66, status=FAILED, diagnostics=Vertex failed, vertexName=CFEA911CACD2465C9FB160575547F456, vertexId=vertex_1463524601480_0001_1_00, diagnostics=[Task failed, taskId=task_1463524601480_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524601480_0001_1_00 [CFEA911CACD2465C9FB160575547F456] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:36:41,711 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524601480_0001_1 finished with state: FAILED
2016-05-17 22:36:41,711 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524601480_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:41,711 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524601480_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:36:41 Completed Dag: dag_1463524601480_0001_1
2016-05-17 22:36:41,712 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524601480_0001_1, dagState=FAILED
2016-05-17 22:36:41,712 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524601480_0001_1_post
2016-05-17 22:36:41,712 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:41,712 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:41,712 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524601480_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:36:41,709 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=CFEA911CACD2465C9FB160575547F456, sourceVertex=73CBA8433CD6411AB1655CA54808855C
2016-05-17 22:36:41,708 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:41,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=CFEA911CACD2465C9FB160575547F456, sourceVertex=FEF5163C5F464B95A38C141510C20C51
2016-05-17 22:36:41,713 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524601480_0001_00_000001
2016-05-17 22:36:41,713 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524601480_0001_00_000001
2016-05-17 22:36:41,713 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:41,713 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:41,714 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524601480_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:36:41,714 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524601480_0001_00_000001 completed but with errors
2016-05-17 22:36:41,715 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524601480_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:36:41,715 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524601480_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:36:41,715 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524601480_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:36:41,719 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...mblyplatform/0_VALUES_ALL, applicationId=application_1463524601480_0001
2016-05-17 22:36:41,720 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:41,720 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:41,720 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:41,720 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:41,720 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:41,721 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:41,721 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:41,721 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:41,721 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:41,722 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:41,722 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:41,722 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:41,722 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:41,730 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D661A8B9D5774EC9A66C94456402C76A/F5E33BD7A40C45D5AEECAE1F2B8B588E/.tez/application_1463524601480_0001
2016-05-17 22:36:41,730 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping all jobs
2016-05-17 22:36:41,730 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping: (1/1) ...mblyplatform/0_VALUES_ALL
2016-05-17 22:36:41,730 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopped all jobs
2016-05-17 22:36:41,731 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_ALL/_temporary
2016-05-17 22:36:41,731 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.268
2016-05-17 22:36:41,731 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_VALUES_ALL
2016-05-17 22:36:41,752 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:41,752 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:41,780 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 22:36:41,784 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 22:36:41,785 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:41,785 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:41,785 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:41,804 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:41,805 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_REPLACE
2016-05-17 22:36:41,817 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:41,817 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:41,818 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_REPLACE"]
2016-05-17 22:36:41,818 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:41,818 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:41,819 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:41,820 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...platform/0_VALUES_REPLACE
2016-05-17 22:36:41,820 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:41,838 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:41,843 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:41,843 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:41,844 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:41,854 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/24744DB3479449DEADDFB1A50568CF43/DDAAA27DB79D49FAAE859D03182F6E7D/.tez/application_1463524601843_0001 doesn't exist and is created
2016-05-17 22:36:41,871 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:41,871 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:41,872 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/24744DB3479449DEADDFB1A50568CF43/DDAAA27DB79D49FAAE859D03182F6E7D/.tez/application_1463524601843_0001
2016-05-17 22:36:41,876 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524601843_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:41,886 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:41,887 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:41,890 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:41,890 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:41,890 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/24744DB3479449DEADDFB1A50568CF43/DDAAA27DB79D49FAAE859D03182F6E7D/.tez/application_1463524601843_0001/localmode-log-dir
2016-05-17 22:36:41,891 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/24744DB3479449DEADDFB1A50568CF43/DDAAA27DB79D49FAAE859D03182F6E7D/.tez/application_1463524601843_0001/localmode-log-dir/history.txt.appattempt_1463524601843_0001_000000, maxErrors=10
2016-05-17 22:36:41,891 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:41,891 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524601843_0001_000000, appSubmitTime=1463524601876, launchTime=1463524601876
2016-05-17 22:36:41,894 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:41,895 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:41,895 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:41,895 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:41,905 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49556
2016-05-17 22:36:41,907 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:41,909 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524601843_0001_000000, startTime=1463524601909
2016-05-17 22:36:41,909 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:41,909 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:41,972 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:41,972 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:41,972 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/0_VALUES_REPLACE, applicationId=application_1463524601843_0001, dagName=[24744DB3479449DEADDFB1A50568CF43/DDAAA27DB79D49FAAE859D03182F6E7D] each/(1/1) ...platform/0_VALUES_REPLACE
2016-05-17 22:36:41,986 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [24744DB3479449DEADDFB1A50568CF43/DDAAA27DB79D49FAAE859D03182F6E7D] each/(1/1) ...platform/0_VALUES_REPLACE
2016-05-17 22:36:41,987 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524601843_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/24744DB3479449DEADDFB1A50568CF43/DDAAA27DB79D49FAAE859D03182F6E7D/.tez/application_1463524601843_0001/localmode-log-dir/dag_1463524601843_0001_1.dot
2016-05-17 22:36:41,988 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/24744DB3479449DEADDFB1A50568CF43/DDAAA27DB79D49FAAE859D03182F6E7D/.tez/application_1463524601843_0001/localmode-log-dir/dag_1463524601843_0001_1-tez-dag.pb.txt
2016-05-17 22:36:41,991 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524601843_0001_1
2016-05-17 22:36:41,992 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:41,992 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [24744DB3479449DEADDFB1A50568CF43/DDAAA27DB79D49FAAE859D03182F6E7D] each/(1/1) ...platform/0_VALUES_REPLACE
2016-05-17 22:36:41 Running Dag: dag_1463524601843_0001_1
2016-05-17 22:36:42,004 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:42,004 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601843_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524601843_0001_1, submitTime=1463524601986
2016-05-17 22:36:42,004 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F74B455DC8B04C31ADAC0D1D31CF91F6
2016-05-17 22:36:42,005 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:42,005 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:42,006 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601843_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524601843_0001_1, initTime=1463524602004
2016-05-17 22:36:42,006 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524601843_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:42,006 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601843_0001_1][Event:DAG_STARTED]: dagID=dag_1463524601843_0001_1, startTime=1463524602006
2016-05-17 22:36:42,006 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:42,006 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524601843_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:42,006 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F74B455DC8B04C31ADAC0D1D31CF91F6 : {84F1FB32C6CD4401B7E70DA1F994B0B1={InputName=84F1FB32C6CD4401B7E70DA1F994B0B1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:42,006 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 84F1FB32C6CD4401B7E70DA1F994B0B1, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:42,006 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6]
2016-05-17 22:36:42,006 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6]
2016-05-17 22:36:42,007 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6]
2016-05-17 22:36:42,007 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6]. Starting root input initializers: 1
2016-05-17 22:36:42,007 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:42,007 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:42,007 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 84F1FB32C6CD4401B7E70DA1F994B0B1 on vertex vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6]
2016-05-17 22:36:42,008 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/0_VALUES_REPLACE, applicationId=application_1463524601843_0001, dagName=[24744DB3479449DEADDFB1A50568CF43/DDAAA27DB79D49FAAE859D03182F6E7D] each/(1/1) ...platform/0_VALUES_REPLACE
2016-05-17 22:36:42,008 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:42,008 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:42,010 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:42,011 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 84F1FB32C6CD4401B7E70DA1F994B0B1 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:42,011 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:42,012 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:42,014 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:42,014 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 84F1FB32C6CD4401B7E70DA1F994B0B1 on vertex vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6]
2016-05-17 22:36:42,014 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:42,015 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {84F1FB32C6CD4401B7E70DA1F994B0B1=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:42,015 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6] parallelism set to 1
2016-05-17 22:36:42,015 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:42,015 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601843_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524601843_0001_1_00, eventCount=1
2016-05-17 22:36:42,015 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6]
2016-05-17 22:36:42,015 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=0AF056BFEFE44B3380E644A7C543C52D, vertexId=vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:42,015 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=0AF056BFEFE44B3380E644A7C543C52D, vertexId=vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6]
2016-05-17 22:36:42,016 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=0AF056BFEFE44B3380E644A7C543C52D, vertexName=F74B455DC8B04C31ADAC0D1D31CF91F6, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:42,017 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=0AF056BFEFE44B3380E644A7C543C52D, vertexId=vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6]
2016-05-17 22:36:42,017 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601843_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F74B455DC8B04C31ADAC0D1D31CF91F6, vertexId=vertex_1463524601843_0001_1_00, initRequestedTime=1463524602006, initedTime=1463524602017, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:42,017 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6] with distanceFromRoot: 0
2016-05-17 22:36:42,017 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:42,018 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:42,018 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F74B455DC8B04C31ADAC0D1D31CF91F6
2016-05-17 22:36:42,018 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6]
2016-05-17 22:36:42,018 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:42,018 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601843_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524601843_0001_1_00, startRequestedTime=1463524602018, startedTime=1463524602018
2016-05-17 22:36:42,019 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:42,019 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:42,019 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601843_0001_1][Event:TASK_STARTED]: vertexName=F74B455DC8B04C31ADAC0D1D31CF91F6, taskId=task_1463524601843_0001_1_00_000000, scheduledTime=1463524602019, launchTime=1463524602019
2016-05-17 22:36:42,019 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524601843_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:42,019 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6]
2016-05-17 22:36:42,020 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [24744DB3479449DEADDFB1A50568CF43/DDAAA27DB79D49FAAE859D03182F6E7D] each/(1/1) ...platform/0_VALUES_REPLACE, VertexName: F74B455DC8B04C31ADAC0D1D31CF91F6, VertexParallelism: 1, TaskAttemptID:attempt_1463524601843_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=84F1FB32C6CD4401B7E70DA1F994B0B1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=0AF056BFEFE44B3380E644A7C543C52D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:42,020 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524601843_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:42,020 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:42,021 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:42,021 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:42,021 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:42,021 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524601843_0001_00_000001
2016-05-17 22:36:42,022 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524601843_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:42,023 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524601843_0001_00_000001
2016-05-17 22:36:42,023 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:42,023 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524601843_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:42,024 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:42,024 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601843_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524601843_0001_00_000001, launchTime=1463524602023
2016-05-17 22:36:42,024 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524601843_0001_00_000001
2016-05-17 22:36:42,025 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524601843_0001_1_00_000000_0] to container: [container_1463524601843_0001_00_000001]
2016-05-17 22:36:42,025 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524601843_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:42,025 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524601843_0001_1_00_000000_0] started. Is using containerId: [container_1463524601843_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:42,026 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601843_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F74B455DC8B04C31ADAC0D1D31CF91F6, taskAttemptId=attempt_1463524601843_0001_1_00_000000_0, startTime=1463524602025, containerId=container_1463524601843_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524601843_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:42,026 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524601843_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:42,026 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524601843_0001, with dag id: dag_1463524601843_0001_1
2016-05-17 22:36:42,026 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524601843_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:42,026 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:42,027 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524601843_0001_00_000001 given task: attempt_1463524601843_0001_1_00_000000_0
2016-05-17 22:36:42,029 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524601843_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524601843_0001_1_00_000000_0
2016-05-17 22:36:42 Starting to run new task attempt: attempt_1463524601843_0001_1_00_000000_0
2016-05-17 22:36:42,030 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524601843_0001_1_00_000000_0
2016-05-17 22:36:42,030 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:42,030 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:42,030 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:42,030 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:42,030 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:42,031 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:42,031 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [24744DB3479449DEADDFB1A50568CF43/DDAAA27DB79D49FAAE859D03182F6E7D] each/(1/1) ...platform/0_VALUES_REPLACE, VertexName: F74B455DC8B04C31ADAC0D1D31CF91F6, VertexParallelism: 1, TaskAttemptID:attempt_1463524601843_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=84F1FB32C6CD4401B7E70DA1F994B0B1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=0AF056BFEFE44B3380E644A7C543C52D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:42,031 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:42,033 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524601843_0001_1_00_000000_0
2016-05-17 22:36:42,033 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:42,033 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:42,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=84F1FB32C6CD4401B7E70DA1F994B0B1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:42,039 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524601843_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:42,040 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=0AF056BFEFE44B3380E644A7C543C52D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:42,034 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:42,040 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:42,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 84F1FB32C6CD4401B7E70DA1F994B0B1
2016-05-17 22:36:42,041 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 84F1FB32C6CD4401B7E70DA1F994B0B1
2016-05-17 22:36:42,042 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:42,042 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 0AF056BFEFE44B3380E644A7C543C52D
2016-05-17 22:36:42,042 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:42,042 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 0AF056BFEFE44B3380E644A7C543C52D
2016-05-17 22:36:42,043 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:42,044 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:42,045 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 84F1FB32C6CD4401B7E70DA1F994B0B1
2016-05-17 22:36:42,045 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 84F1FB32C6CD4401B7E70DA1F994B0B1
2016-05-17 22:36:42,050 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F74B455DC8B04C31ADAC0D1D31CF91F6, ordinal: 0
2016-05-17 22:36:42,051 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F74B455DC8B04C31ADAC0D1D31CF91F6, mem on start (mb), free: 202, total: 364, max: 672
2016-05-17 22:36:42,051 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:42,051 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:42,051 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:42,052 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:42,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 0AF056BFEFE44B3380E644A7C543C52D
2016-05-17 22:36:42,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:42,052 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:42,052 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:42,052 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:42,052 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:42,052 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:42,052 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:42,053 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:42,053 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 84F1FB32C6CD4401B7E70DA1F994B0B1, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:42,053 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 0AF056BFEFE44B3380E644A7C543C52D, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:42,053 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:42,053 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 84F1FB32C6CD4401B7E70DA1F994B0B1
2016-05-17 22:36:42,053 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:42,054 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 84F1FB32C6CD4401B7E70DA1F994B0B1
2016-05-17 22:36:42,054 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 84F1FB32C6CD4401B7E70DA1F994B0B1 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:42,054 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:42,054 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:42,055 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:42,055 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524601843_0001_1_00_000000_0
2016-05-17 22:36:42,055 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:42,055 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7d6331fc
2016-05-17 22:36:42,057 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:42,058 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 84F1FB32C6CD4401B7E70DA1F994B0B1
2016-05-17 22:36:42,058 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_REPLACE"], id: 0AF056BFEFE44B3380E644A7C543C52D
2016-05-17 22:36:42,058 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_REPLACE"]
2016-05-17 22:36:42,059 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:42,059 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:42,060 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7d6331fc
2016-05-17 22:36:42,060 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:42,060 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:42,061 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F74B455DC8B04C31ADAC0D1D31CF91F6, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:42,062 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:42,067 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:42,072 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:42,072 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:42,072 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524601843_0001_1_00_000000_0
2016-05-17 22:36:42,073 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524601843_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:42,073 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524601843_0001_1_00_000000_0
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:42,079 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524602061, Process_Duration=12, Process_End_Time=1463524602073, Read_Duration=1, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:36:42,079 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601843_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F74B455DC8B04C31ADAC0D1D31CF91F6, taskAttemptId=attempt_1463524601843_0001_1_00_000000_0, startTime=1463524602025, finishTime=1463524602079, timeTaken=54, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524602061, Process_Duration=12, Process_End_Time=1463524602073, Read_Duration=1, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:36:42,079 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:42,079 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524601843_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:36:42,080 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524601843_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:36:42,080 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F74B455DC8B04C31ADAC0D1D31CF91F6, mem on close (mb), free: 201, total: 364, max: 672
2016-05-17 22:36:42,080 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601843_0001_1][Event:TASK_FINISHED]: vertexName=F74B455DC8B04C31ADAC0D1D31CF91F6, taskId=task_1463524601843_0001_1_00_000000, startTime=1463524602025, finishTime=1463524602080, timeTaken=55, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
], counters=Counters: 0
2016-05-17 22:36:42,080 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524601843_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:36:42,080 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6] : 1
2016-05-17 22:36:42,080 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6] because task failed: task_1463524601843_0001_1_00_000000
2016-05-17 22:36:42,080 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:36:42,081 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:36:42,081 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6]
2016-05-17 22:36:42,081 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=0AF056BFEFE44B3380E644A7C543C52D, vertexId=vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6]
2016-05-17 22:36:42,080 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:42,081 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:42,084 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:42,084 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601843_0001_1][Event:VERTEX_FINISHED]: vertexName=F74B455DC8B04C31ADAC0D1D31CF91F6, vertexId=vertex_1463524601843_0001_1_00, initRequestedTime=1463524602006, initedTime=1463524602017, startRequestedTime=1463524602018, startedTime=1463524602018, finishTime=1463524602084, timeTaken=66, status=FAILED, diagnostics=Task failed, taskId=task_1463524601843_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524602025, firstTasksToStart=[ task_1463524601843_0001_1_00_000000 ], lastTaskFinishTime=1463524602079, lastTasksToFinish=[ task_1463524601843_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:36:42,084 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:36:42,084 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601843_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524601843_0001_00_000001, stoppedTime=1463524602084, exitStatus=0
2016-05-17 22:36:42,084 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:42,084 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524601843_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:42,085 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524601843_0001_1_00_000000_0 failed: true container: container_1463524601843_0001_00_000001 numFailedTAs: 0
2016-05-17 22:36:42,085 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:36:42,085 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:36:42,085 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524601843_0001_1
2016-05-17 22:36:42,085 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 0AF056BFEFE44B3380E644A7C543C52D for vertex: vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6]
2016-05-17 22:36:42,086 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524601843_0001_1, queueSize=0
2016-05-17 22:36:42,086 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:42,086 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=F74B455DC8B04C31ADAC0D1D31CF91F6, index=0
2016-05-17 22:36:42,086 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=F74B455DC8B04C31ADAC0D1D31CF91F6, sourceVertex=84F1FB32C6CD4401B7E70DA1F994B0B1
2016-05-17 22:36:42,086 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=F74B455DC8B04C31ADAC0D1D31CF91F6, sourceVertex=0AF056BFEFE44B3380E644A7C543C52D
2016-05-17 22:36:42,086 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524601843_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524601843_0001_1, startTime=1463524602006, finishTime=1463524602085, timeTaken=79, status=FAILED, diagnostics=Vertex failed, vertexName=F74B455DC8B04C31ADAC0D1D31CF91F6, vertexId=vertex_1463524601843_0001_1_00, diagnostics=[Task failed, taskId=task_1463524601843_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524601843_0001_1_00 [F74B455DC8B04C31ADAC0D1D31CF91F6] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:36:42,086 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524601843_0001_1 finished with state: FAILED
2016-05-17 22:36:42,087 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524601843_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:42,087 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524601843_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:36:42 Completed Dag: dag_1463524601843_0001_1
2016-05-17 22:36:42,087 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524601843_0001_1, dagState=FAILED
2016-05-17 22:36:42,087 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524601843_0001_1_post
2016-05-17 22:36:42,088 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:42,088 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:42,088 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524601843_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:36:42,088 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524601843_0001_00_000001
2016-05-17 22:36:42,088 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524601843_0001_00_000001
2016-05-17 22:36:42,089 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524601843_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:36:42,089 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524601843_0001_00_000001 completed but with errors
2016-05-17 22:36:42,090 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524601843_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:36:42,090 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524601843_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:36:42,090 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524601843_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:36:42,093 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/0_VALUES_REPLACE, applicationId=application_1463524601843_0001
2016-05-17 22:36:42,095 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:42,095 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:42,095 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:42,095 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:42,095 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:42,096 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:42,096 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:42,096 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:42,096 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:42,096 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:42,097 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:42,097 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:42,097 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:42,102 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/24744DB3479449DEADDFB1A50568CF43/DDAAA27DB79D49FAAE859D03182F6E7D/.tez/application_1463524601843_0001
2016-05-17 22:36:42,102 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping all jobs
2016-05-17 22:36:42,102 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping: (1/1) ...platform/0_VALUES_REPLACE
2016-05-17 22:36:42,102 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopped all jobs
2016-05-17 22:36:42,103 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_REPLACE/_temporary
2016-05-17 22:36:42,103 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.286
2016-05-17 22:36:42,103 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_VALUES_REPLACE
2016-05-17 22:36:42,124 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:42,124 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:42,152 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-05-17 22:36:42,164 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-17 22:36:42,164 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:42,164 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:42,164 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:42,177 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:42,177 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_SWAP
2016-05-17 22:36:42,185 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:42,185 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:42,185 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_SWAP"]
2016-05-17 22:36:42,186 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:42,186 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:42,186 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:42,187 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...blyplatform/0_VALUES_SWAP
2016-05-17 22:36:42,187 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:42,198 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:42,203 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:42,203 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:42,203 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:42,213 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CA1843519E3B4B409402796F2C1E2B5A/E67D6DB1F8234DF9BDEE809370CF5649/.tez/application_1463524602203_0001 doesn't exist and is created
2016-05-17 22:36:42,229 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:42,229 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:42,229 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CA1843519E3B4B409402796F2C1E2B5A/E67D6DB1F8234DF9BDEE809370CF5649/.tez/application_1463524602203_0001
2016-05-17 22:36:42,232 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524602203_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:42,245 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:42,246 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:42,247 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:42,247 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:42,247 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CA1843519E3B4B409402796F2C1E2B5A/E67D6DB1F8234DF9BDEE809370CF5649/.tez/application_1463524602203_0001/localmode-log-dir
2016-05-17 22:36:42,248 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CA1843519E3B4B409402796F2C1E2B5A/E67D6DB1F8234DF9BDEE809370CF5649/.tez/application_1463524602203_0001/localmode-log-dir/history.txt.appattempt_1463524602203_0001_000000, maxErrors=10
2016-05-17 22:36:42,248 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:42,248 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524602203_0001_000000, appSubmitTime=1463524602231, launchTime=1463524602231
2016-05-17 22:36:42,251 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:42,252 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:42,253 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:42,253 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:42,255 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:57550
2016-05-17 22:36:42,257 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:42,258 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524602203_0001_000000, startTime=1463524602258
2016-05-17 22:36:42,258 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:42,258 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:42,329 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:42,330 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:42,330 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/0_VALUES_SWAP, applicationId=application_1463524602203_0001, dagName=[CA1843519E3B4B409402796F2C1E2B5A/E67D6DB1F8234DF9BDEE809370CF5649] each/(1/1) ...blyplatform/0_VALUES_SWAP
2016-05-17 22:36:42,344 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [CA1843519E3B4B409402796F2C1E2B5A/E67D6DB1F8234DF9BDEE809370CF5649] each/(1/1) ...blyplatform/0_VALUES_SWAP
2016-05-17 22:36:42,345 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524602203_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CA1843519E3B4B409402796F2C1E2B5A/E67D6DB1F8234DF9BDEE809370CF5649/.tez/application_1463524602203_0001/localmode-log-dir/dag_1463524602203_0001_1.dot
2016-05-17 22:36:42,345 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CA1843519E3B4B409402796F2C1E2B5A/E67D6DB1F8234DF9BDEE809370CF5649/.tez/application_1463524602203_0001/localmode-log-dir/dag_1463524602203_0001_1-tez-dag.pb.txt
2016-05-17 22:36:42,349 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524602203_0001_1
2016-05-17 22:36:42,349 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:42,349 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [CA1843519E3B4B409402796F2C1E2B5A/E67D6DB1F8234DF9BDEE809370CF5649] each/(1/1) ...blyplatform/0_VALUES_SWAP
2016-05-17 22:36:42 Running Dag: dag_1463524602203_0001_1
2016-05-17 22:36:42,360 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:42,360 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602203_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524602203_0001_1, submitTime=1463524602344
2016-05-17 22:36:42,361 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D5E5597E101D4B39B16505801CFFBD12
2016-05-17 22:36:42,361 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:42,361 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:42,361 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602203_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524602203_0001_1, initTime=1463524602361
2016-05-17 22:36:42,361 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524602203_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:42,362 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602203_0001_1][Event:DAG_STARTED]: dagID=dag_1463524602203_0001_1, startTime=1463524602361
2016-05-17 22:36:42,362 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:42,362 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524602203_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:42,362 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/0_VALUES_SWAP, applicationId=application_1463524602203_0001, dagName=[CA1843519E3B4B409402796F2C1E2B5A/E67D6DB1F8234DF9BDEE809370CF5649] each/(1/1) ...blyplatform/0_VALUES_SWAP
2016-05-17 22:36:42,362 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:42,362 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D5E5597E101D4B39B16505801CFFBD12 : {82051684FE4B46BCB6BD185570D7093E={InputName=82051684FE4B46BCB6BD185570D7093E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:42,362 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 82051684FE4B46BCB6BD185570D7093E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:42,362 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12]
2016-05-17 22:36:42,363 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12]
2016-05-17 22:36:42,363 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12]
2016-05-17 22:36:42,363 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12]. Starting root input initializers: 1
2016-05-17 22:36:42,364 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:42,364 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 82051684FE4B46BCB6BD185570D7093E on vertex vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12]
2016-05-17 22:36:42,366 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:42,366 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 82051684FE4B46BCB6BD185570D7093E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:42,367 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:42,367 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:42,368 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:42,370 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 82051684FE4B46BCB6BD185570D7093E on vertex vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12]
2016-05-17 22:36:42,370 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:42,370 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {82051684FE4B46BCB6BD185570D7093E=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:42,370 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12] parallelism set to 1
2016-05-17 22:36:42,371 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:42,371 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602203_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524602203_0001_1_00, eventCount=1
2016-05-17 22:36:42,371 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12]
2016-05-17 22:36:42,371 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=3BFB1CECDA70400D833BAF5D64FE3FDB, vertexId=vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:42,371 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=3BFB1CECDA70400D833BAF5D64FE3FDB, vertexId=vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12]
2016-05-17 22:36:42,373 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=3BFB1CECDA70400D833BAF5D64FE3FDB, vertexName=D5E5597E101D4B39B16505801CFFBD12, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:42,373 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=3BFB1CECDA70400D833BAF5D64FE3FDB, vertexId=vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12]
2016-05-17 22:36:42,374 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602203_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D5E5597E101D4B39B16505801CFFBD12, vertexId=vertex_1463524602203_0001_1_00, initRequestedTime=1463524602362, initedTime=1463524602374, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:42,375 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:42,375 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12] with distanceFromRoot: 0
2016-05-17 22:36:42,375 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:42,376 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D5E5597E101D4B39B16505801CFFBD12
2016-05-17 22:36:42,376 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12]
2016-05-17 22:36:42,377 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602203_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524602203_0001_1_00, startRequestedTime=1463524602376, startedTime=1463524602376
2016-05-17 22:36:42,377 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:42,377 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:42,378 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602203_0001_1][Event:TASK_STARTED]: vertexName=D5E5597E101D4B39B16505801CFFBD12, taskId=task_1463524602203_0001_1_00_000000, scheduledTime=1463524602378, launchTime=1463524602378
2016-05-17 22:36:42,378 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:42,378 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524602203_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:42,379 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12]
2016-05-17 22:36:42,379 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [CA1843519E3B4B409402796F2C1E2B5A/E67D6DB1F8234DF9BDEE809370CF5649] each/(1/1) ...blyplatform/0_VALUES_SWAP, VertexName: D5E5597E101D4B39B16505801CFFBD12, VertexParallelism: 1, TaskAttemptID:attempt_1463524602203_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=82051684FE4B46BCB6BD185570D7093E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=3BFB1CECDA70400D833BAF5D64FE3FDB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:42,380 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524602203_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:42,380 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524602203_0001, with dag id: dag_1463524602203_0001_1
2016-05-17 22:36:42,380 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:42,381 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:42,381 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:42,382 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:42,382 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524602203_0001_00_000001
2016-05-17 22:36:42,382 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524602203_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:42,384 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524602203_0001_00_000001
2016-05-17 22:36:42,384 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:42,385 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524602203_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:42,385 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:42,386 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602203_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524602203_0001_00_000001, launchTime=1463524602385
2016-05-17 22:36:42,386 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524602203_0001_00_000001
2016-05-17 22:36:42,387 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524602203_0001_1_00_000000_0] to container: [container_1463524602203_0001_00_000001]
2016-05-17 22:36:42,387 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524602203_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:42,387 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524602203_0001_00_000001 given task: attempt_1463524602203_0001_1_00_000000_0
2016-05-17 22:36:42,387 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524602203_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524602203_0001_1_00_000000_0
2016-05-17 22:36:42,387 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524602203_0001_1_00_000000_0] started. Is using containerId: [container_1463524602203_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:42 Starting to run new task attempt: attempt_1463524602203_0001_1_00_000000_0
2016-05-17 22:36:42,387 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524602203_0001_1_00_000000_0
2016-05-17 22:36:42,388 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:42,388 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:42,388 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:42,388 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:42,388 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602203_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D5E5597E101D4B39B16505801CFFBD12, taskAttemptId=attempt_1463524602203_0001_1_00_000000_0, startTime=1463524602387, containerId=container_1463524602203_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524602203_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:42,388 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:42,388 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524602203_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:42,388 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:42,388 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524602203_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:42,388 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:42,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [CA1843519E3B4B409402796F2C1E2B5A/E67D6DB1F8234DF9BDEE809370CF5649] each/(1/1) ...blyplatform/0_VALUES_SWAP, VertexName: D5E5597E101D4B39B16505801CFFBD12, VertexParallelism: 1, TaskAttemptID:attempt_1463524602203_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=82051684FE4B46BCB6BD185570D7093E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=3BFB1CECDA70400D833BAF5D64FE3FDB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:42,389 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:42,389 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524602203_0001_1_00_000000_0
2016-05-17 22:36:42,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:42,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:42,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=82051684FE4B46BCB6BD185570D7093E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:42,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:42,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 82051684FE4B46BCB6BD185570D7093E
2016-05-17 22:36:42,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:42,390 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 82051684FE4B46BCB6BD185570D7093E
2016-05-17 22:36:42,392 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:42,392 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 82051684FE4B46BCB6BD185570D7093E
2016-05-17 22:36:42,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 82051684FE4B46BCB6BD185570D7093E
2016-05-17 22:36:42,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=3BFB1CECDA70400D833BAF5D64FE3FDB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:42,396 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:42,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:42,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 3BFB1CECDA70400D833BAF5D64FE3FDB
2016-05-17 22:36:42,397 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:42,398 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 3BFB1CECDA70400D833BAF5D64FE3FDB
2016-05-17 22:36:42,396 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524602203_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:42,404 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D5E5597E101D4B39B16505801CFFBD12, ordinal: 0
2016-05-17 22:36:42,405 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5E5597E101D4B39B16505801CFFBD12, mem on start (mb), free: 142, total: 364, max: 672
2016-05-17 22:36:42,405 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:42,405 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:42,405 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:42,412 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:42,413 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 3BFB1CECDA70400D833BAF5D64FE3FDB
2016-05-17 22:36:42,413 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:42,413 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:42,413 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:42,413 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:42,413 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:42,413 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:42,414 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:42,414 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:42,414 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 82051684FE4B46BCB6BD185570D7093E, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:42,414 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 3BFB1CECDA70400D833BAF5D64FE3FDB, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:42,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:42,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 82051684FE4B46BCB6BD185570D7093E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:42,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:42,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 82051684FE4B46BCB6BD185570D7093E
2016-05-17 22:36:42,414 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:42,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 82051684FE4B46BCB6BD185570D7093E
2016-05-17 22:36:42,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:42,415 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:42,415 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524602203_0001_1_00_000000_0
2016-05-17 22:36:42,415 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:42,416 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@13801d70
2016-05-17 22:36:42,416 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@13801d70
2016-05-17 22:36:42,417 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:42,417 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:42,418 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:42,418 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 82051684FE4B46BCB6BD185570D7093E
2016-05-17 22:36:42,418 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_SWAP"], id: 3BFB1CECDA70400D833BAF5D64FE3FDB
2016-05-17 22:36:42,419 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_SWAP"]
2016-05-17 22:36:42,419 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:42,419 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:42,419 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D5E5597E101D4B39B16505801CFFBD12, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:42,420 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:42,420 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:42,420 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:42,420 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:42,421 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524602203_0001_1_00_000000_0
2016-05-17 22:36:42,421 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524602203_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:42,422 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524602203_0001_1_00_000000_0
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:42,425 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524602419, Process_Duration=2, Process_End_Time=1463524602421, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:36:42,426 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5E5597E101D4B39B16505801CFFBD12, mem on close (mb), free: 140, total: 364, max: 672
2016-05-17 22:36:42,426 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602203_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D5E5597E101D4B39B16505801CFFBD12, taskAttemptId=attempt_1463524602203_0001_1_00_000000_0, startTime=1463524602387, finishTime=1463524602425, timeTaken=38, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524602419, Process_Duration=2, Process_End_Time=1463524602421, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:36:42,426 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524602203_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:36:42,426 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:42,426 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524602203_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:36:42,427 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602203_0001_1][Event:TASK_FINISHED]: vertexName=D5E5597E101D4B39B16505801CFFBD12, taskId=task_1463524602203_0001_1_00_000000, startTime=1463524602387, finishTime=1463524602427, timeTaken=40, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
], counters=Counters: 0
2016-05-17 22:36:42,427 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524602203_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:36:42,427 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:42,427 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12] : 1
2016-05-17 22:36:42,427 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:42,427 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12] because task failed: task_1463524602203_0001_1_00_000000
2016-05-17 22:36:42,427 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:36:42,427 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:36:42,428 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12]
2016-05-17 22:36:42,428 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=3BFB1CECDA70400D833BAF5D64FE3FDB, vertexId=vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12]
2016-05-17 22:36:42,428 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=D5E5597E101D4B39B16505801CFFBD12, index=0
2016-05-17 22:36:42,429 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=D5E5597E101D4B39B16505801CFFBD12, sourceVertex=82051684FE4B46BCB6BD185570D7093E
2016-05-17 22:36:42,429 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=D5E5597E101D4B39B16505801CFFBD12, sourceVertex=3BFB1CECDA70400D833BAF5D64FE3FDB
2016-05-17 22:36:42,429 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:42,429 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524602203_0001_00_000001
2016-05-17 22:36:42,429 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524602203_0001_00_000001
2016-05-17 22:36:42,429 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602203_0001_1][Event:VERTEX_FINISHED]: vertexName=D5E5597E101D4B39B16505801CFFBD12, vertexId=vertex_1463524602203_0001_1_00, initRequestedTime=1463524602362, initedTime=1463524602374, startRequestedTime=1463524602376, startedTime=1463524602376, finishTime=1463524602429, timeTaken=53, status=FAILED, diagnostics=Task failed, taskId=task_1463524602203_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524602387, firstTasksToStart=[ task_1463524602203_0001_1_00_000000 ], lastTaskFinishTime=1463524602425, lastTasksToFinish=[ task_1463524602203_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:36:42,430 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:36:42,430 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602203_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524602203_0001_00_000001, stoppedTime=1463524602430, exitStatus=0
2016-05-17 22:36:42,430 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524602203_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:42,430 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524602203_0001_1_00_000000_0 failed: true container: container_1463524602203_0001_00_000001 numFailedTAs: 0
2016-05-17 22:36:42,430 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:36:42,431 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:42,431 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:36:42,431 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524602203_0001_1
2016-05-17 22:36:42,431 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 3BFB1CECDA70400D833BAF5D64FE3FDB for vertex: vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12]
2016-05-17 22:36:42,431 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524602203_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:36:42,432 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524602203_0001_00_000001 completed but with errors
2016-05-17 22:36:42,432 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524602203_0001_1, queueSize=0
2016-05-17 22:36:42,432 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:42,433 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602203_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524602203_0001_1, startTime=1463524602361, finishTime=1463524602432, timeTaken=71, status=FAILED, diagnostics=Vertex failed, vertexName=D5E5597E101D4B39B16505801CFFBD12, vertexId=vertex_1463524602203_0001_1_00, diagnostics=[Task failed, taskId=task_1463524602203_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524602203_0001_1_00 [D5E5597E101D4B39B16505801CFFBD12] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:36:42,433 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524602203_0001_1 finished with state: FAILED
2016-05-17 22:36:42,433 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524602203_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:42,433 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524602203_0001_00_000001
2016-05-17 22:36:42,433 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524602203_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:36:42,433 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524602203_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:42,434 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:42,434 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:42 Completed Dag: dag_1463524602203_0001_1
2016-05-17 22:36:42,434 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524602203_0001_1, dagState=FAILED
2016-05-17 22:36:42,434 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524602203_0001_1_post
2016-05-17 22:36:42,434 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:42,435 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:42,435 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524602203_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:36:42,441 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/0_VALUES_SWAP, applicationId=application_1463524602203_0001
2016-05-17 22:36:42,443 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:42,443 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:42,443 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:42,443 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:42,443 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:42,443 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:42,444 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:42,444 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:42,444 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:42,444 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:42,445 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:42,445 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:42,445 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:42,450 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CA1843519E3B4B409402796F2C1E2B5A/E67D6DB1F8234DF9BDEE809370CF5649/.tez/application_1463524602203_0001
2016-05-17 22:36:42,450 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping all jobs
2016-05-17 22:36:42,451 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping: (1/1) ...blyplatform/0_VALUES_SWAP
2016-05-17 22:36:42,451 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopped all jobs
2016-05-17 22:36:42,451 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_VALUES_SWAP/_temporary
2016-05-17 22:36:42,452 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.266
2016-05-17 22:36:42,452 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_VALUES_SWAP
2016-05-17 22:36:42,483 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:42,483 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:42,517 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-17 22:36:42,521 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 22:36:42,521 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:42,521 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:42,521 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:42,535 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:42,536 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_ARGS_-1
2016-05-17 22:36:42,549 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:42,549 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:42,549 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_ARGS_-1"]
2016-05-17 22:36:42,549 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:42,549 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:42,549 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:42,550 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...ssemblyplatform/0_ARGS_-1
2016-05-17 22:36:42,550 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:42,564 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:42,570 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:42,570 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:42,570 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:42,582 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/82E52E9B069647A19266C86D6C8E8004/E066F5946D4A438BA37C7CB9F957FA85/.tez/application_1463524602570_0001 doesn't exist and is created
2016-05-17 22:36:42,600 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:42,600 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:42,601 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/82E52E9B069647A19266C86D6C8E8004/E066F5946D4A438BA37C7CB9F957FA85/.tez/application_1463524602570_0001
2016-05-17 22:36:42,603 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524602570_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:42,627 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:42,628 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:42,629 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:42,629 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:42,629 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/82E52E9B069647A19266C86D6C8E8004/E066F5946D4A438BA37C7CB9F957FA85/.tez/application_1463524602570_0001/localmode-log-dir
2016-05-17 22:36:42,630 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/82E52E9B069647A19266C86D6C8E8004/E066F5946D4A438BA37C7CB9F957FA85/.tez/application_1463524602570_0001/localmode-log-dir/history.txt.appattempt_1463524602570_0001_000000, maxErrors=10
2016-05-17 22:36:42,630 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:42,630 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524602570_0001_000000, appSubmitTime=1463524602603, launchTime=1463524602603
2016-05-17 22:36:42,635 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:38441
2016-05-17 22:36:42,638 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:42,639 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:42,640 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:42,640 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:42,642 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:42,643 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524602570_0001_000000, startTime=1463524602643
2016-05-17 22:36:42,643 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:42,643 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:42,700 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:42,701 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:42,701 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ssemblyplatform/0_ARGS_-1, applicationId=application_1463524602570_0001, dagName=[82E52E9B069647A19266C86D6C8E8004/E066F5946D4A438BA37C7CB9F957FA85] each/(1/1) ...ssemblyplatform/0_ARGS_-1
2016-05-17 22:36:42,716 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [82E52E9B069647A19266C86D6C8E8004/E066F5946D4A438BA37C7CB9F957FA85] each/(1/1) ...ssemblyplatform/0_ARGS_-1
2016-05-17 22:36:42,717 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524602570_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/82E52E9B069647A19266C86D6C8E8004/E066F5946D4A438BA37C7CB9F957FA85/.tez/application_1463524602570_0001/localmode-log-dir/dag_1463524602570_0001_1.dot
2016-05-17 22:36:42,717 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/82E52E9B069647A19266C86D6C8E8004/E066F5946D4A438BA37C7CB9F957FA85/.tez/application_1463524602570_0001/localmode-log-dir/dag_1463524602570_0001_1-tez-dag.pb.txt
2016-05-17 22:36:42,721 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524602570_0001_1
2016-05-17 22:36:42,721 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:42,721 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [82E52E9B069647A19266C86D6C8E8004/E066F5946D4A438BA37C7CB9F957FA85] each/(1/1) ...ssemblyplatform/0_ARGS_-1
2016-05-17 22:36:42 Running Dag: dag_1463524602570_0001_1
2016-05-17 22:36:42,732 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602570_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524602570_0001_1, submitTime=1463524602716
2016-05-17 22:36:42,732 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:42,732 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C135D425603B47868031903B025259F0
2016-05-17 22:36:42,732 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:42,733 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602570_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524602570_0001_1, initTime=1463524602732
2016-05-17 22:36:42,733 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:42,733 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524602570_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:42,734 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602570_0001_1][Event:DAG_STARTED]: dagID=dag_1463524602570_0001_1, startTime=1463524602734
2016-05-17 22:36:42,734 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:42,734 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524602570_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:42,734 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C135D425603B47868031903B025259F0 : {3EE67AD9D86C49D3804AC68CCADFE410={InputName=3EE67AD9D86C49D3804AC68CCADFE410}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:42,734 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 3EE67AD9D86C49D3804AC68CCADFE410, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:42,734 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:42,734 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0]
2016-05-17 22:36:42,734 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0]
2016-05-17 22:36:42,735 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0]
2016-05-17 22:36:42,735 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0]. Starting root input initializers: 1
2016-05-17 22:36:42,735 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ssemblyplatform/0_ARGS_-1, applicationId=application_1463524602570_0001, dagName=[82E52E9B069647A19266C86D6C8E8004/E066F5946D4A438BA37C7CB9F957FA85] each/(1/1) ...ssemblyplatform/0_ARGS_-1
2016-05-17 22:36:42,735 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:42,736 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 3EE67AD9D86C49D3804AC68CCADFE410 on vertex vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0]
2016-05-17 22:36:42,738 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:42,738 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 3EE67AD9D86C49D3804AC68CCADFE410 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:42,739 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:42,739 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:42,740 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:42,740 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 3EE67AD9D86C49D3804AC68CCADFE410 on vertex vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0]
2016-05-17 22:36:42,740 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:42,740 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {3EE67AD9D86C49D3804AC68CCADFE410=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:42,741 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0] parallelism set to 1
2016-05-17 22:36:42,741 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:42,741 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602570_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524602570_0001_1_00, eventCount=1
2016-05-17 22:36:42,741 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0]
2016-05-17 22:36:42,741 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=51C9F67265B5496DB4C324C2B3BBC0C6, vertexId=vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:42,741 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=51C9F67265B5496DB4C324C2B3BBC0C6, vertexId=vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0]
2016-05-17 22:36:42,743 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=51C9F67265B5496DB4C324C2B3BBC0C6, vertexName=C135D425603B47868031903B025259F0, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:42,743 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=51C9F67265B5496DB4C324C2B3BBC0C6, vertexId=vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0]
2016-05-17 22:36:42,744 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:42,744 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602570_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C135D425603B47868031903B025259F0, vertexId=vertex_1463524602570_0001_1_00, initRequestedTime=1463524602734, initedTime=1463524602744, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:42,745 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0] with distanceFromRoot: 0
2016-05-17 22:36:42,745 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:42,745 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C135D425603B47868031903B025259F0
2016-05-17 22:36:42,745 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0]
2016-05-17 22:36:42,745 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:42,745 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602570_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524602570_0001_1_00, startRequestedTime=1463524602745, startedTime=1463524602745
2016-05-17 22:36:42,746 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:42,746 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:42,746 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602570_0001_1][Event:TASK_STARTED]: vertexName=C135D425603B47868031903B025259F0, taskId=task_1463524602570_0001_1_00_000000, scheduledTime=1463524602746, launchTime=1463524602746
2016-05-17 22:36:42,747 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524602570_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:42,747 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0]
2016-05-17 22:36:42,747 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [82E52E9B069647A19266C86D6C8E8004/E066F5946D4A438BA37C7CB9F957FA85] each/(1/1) ...ssemblyplatform/0_ARGS_-1, VertexName: C135D425603B47868031903B025259F0, VertexParallelism: 1, TaskAttemptID:attempt_1463524602570_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3EE67AD9D86C49D3804AC68CCADFE410, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=51C9F67265B5496DB4C324C2B3BBC0C6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:42,747 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524602570_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:42,748 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:42,748 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:42,749 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:42,749 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:42,749 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524602570_0001_00_000001
2016-05-17 22:36:42,749 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524602570_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:42,749 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524602570_0001, with dag id: dag_1463524602570_0001_1
2016-05-17 22:36:42,751 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524602570_0001_00_000001
2016-05-17 22:36:42,751 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:42,752 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524602570_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:42,752 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:42,752 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602570_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524602570_0001_00_000001, launchTime=1463524602752
2016-05-17 22:36:42,753 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524602570_0001_00_000001
2016-05-17 22:36:42,754 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524602570_0001_1_00_000000_0] to container: [container_1463524602570_0001_00_000001]
2016-05-17 22:36:42,754 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524602570_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:42,754 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524602570_0001_00_000001 given task: attempt_1463524602570_0001_1_00_000000_0
2016-05-17 22:36:42,754 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524602570_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524602570_0001_1_00_000000_0
2016-05-17 22:36:42,754 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524602570_0001_1_00_000000_0] started. Is using containerId: [container_1463524602570_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:42,755 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602570_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C135D425603B47868031903B025259F0, taskAttemptId=attempt_1463524602570_0001_1_00_000000_0, startTime=1463524602754, containerId=container_1463524602570_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524602570_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:42 Starting to run new task attempt: attempt_1463524602570_0001_1_00_000000_0
2016-05-17 22:36:42,755 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524602570_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:42,755 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:42,755 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524602570_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:42,755 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524602570_0001_1_00_000000_0
2016-05-17 22:36:42,755 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:42,756 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:42,756 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:42,756 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:42,756 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:42,756 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:42,756 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [82E52E9B069647A19266C86D6C8E8004/E066F5946D4A438BA37C7CB9F957FA85] each/(1/1) ...ssemblyplatform/0_ARGS_-1, VertexName: C135D425603B47868031903B025259F0, VertexParallelism: 1, TaskAttemptID:attempt_1463524602570_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3EE67AD9D86C49D3804AC68CCADFE410, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=51C9F67265B5496DB4C324C2B3BBC0C6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:42,756 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:42,757 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524602570_0001_1_00_000000_0
2016-05-17 22:36:42,757 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:42,757 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:42,757 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524602570_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:42,758 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=51C9F67265B5496DB4C324C2B3BBC0C6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:42,758 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=3EE67AD9D86C49D3804AC68CCADFE410, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:42,757 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:42,759 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:42,759 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 51C9F67265B5496DB4C324C2B3BBC0C6
2016-05-17 22:36:42,759 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:42,759 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 51C9F67265B5496DB4C324C2B3BBC0C6
2016-05-17 22:36:42,761 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:42,761 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:42,761 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 3EE67AD9D86C49D3804AC68CCADFE410
2016-05-17 22:36:42,762 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 3EE67AD9D86C49D3804AC68CCADFE410
2016-05-17 22:36:42,764 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:42,764 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 3EE67AD9D86C49D3804AC68CCADFE410
2016-05-17 22:36:42,764 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 3EE67AD9D86C49D3804AC68CCADFE410
2016-05-17 22:36:42,768 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C135D425603B47868031903B025259F0, ordinal: 0
2016-05-17 22:36:42,768 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C135D425603B47868031903B025259F0, mem on start (mb), free: 269, total: 355, max: 672
2016-05-17 22:36:42,769 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:42,769 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:42,769 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:42,770 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:42,770 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 51C9F67265B5496DB4C324C2B3BBC0C6
2016-05-17 22:36:42,770 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:42,770 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:42,770 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:42,770 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:42,770 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:42,771 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:42,771 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:42,771 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:42,771 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 51C9F67265B5496DB4C324C2B3BBC0C6, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:42,771 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 3EE67AD9D86C49D3804AC68CCADFE410, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:42,771 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:42,771 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 3EE67AD9D86C49D3804AC68CCADFE410 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:42,771 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 3EE67AD9D86C49D3804AC68CCADFE410
2016-05-17 22:36:42,772 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:42,772 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 3EE67AD9D86C49D3804AC68CCADFE410
2016-05-17 22:36:42,772 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:42,772 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:42,772 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:42,772 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524602570_0001_1_00_000000_0
2016-05-17 22:36:42,773 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:42,773 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1bf2885a
2016-05-17 22:36:42,774 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1bf2885a
2016-05-17 22:36:42,774 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:42,774 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:42,775 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:42,776 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 3EE67AD9D86C49D3804AC68CCADFE410
2016-05-17 22:36:42,776 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_ARGS_-1"], id: 51C9F67265B5496DB4C324C2B3BBC0C6
2016-05-17 22:36:42,776 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_ARGS_-1"]
2016-05-17 22:36:42,777 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:42,777 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:42,778 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C135D425603B47868031903B025259F0, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:42,780 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:42,781 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:42,781 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524602570_0001_1_00_000000_0
2016-05-17 22:36:42,781 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524602570_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:42,781 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524602570_0001_1_00_000000_0
2016-05-17 22:36:42,782 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C135D425603B47868031903B025259F0, mem on close (mb), free: 267, total: 355, max: 672
2016-05-17 22:36:42 Completed running task attempt: attempt_1463524602570_0001_1_00_000000_0
2016-05-17 22:36:42,782 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524602570_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:42,783 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:42,783 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602570_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C135D425603B47868031903B025259F0, taskAttemptId=attempt_1463524602570_0001_1_00_000000_0, startTime=1463524602754, finishTime=1463524602783, timeTaken=29, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524602778, Process_Duration=3, Process_End_Time=1463524602781, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:42,783 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524602570_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:42,784 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524602570_0001_1_00_000000_0
2016-05-17 22:36:42,784 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:42,784 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:42,784 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602570_0001_1][Event:TASK_FINISHED]: vertexName=C135D425603B47868031903B025259F0, taskId=task_1463524602570_0001_1_00_000000, startTime=1463524602754, finishTime=1463524602784, timeTaken=30, status=SUCCEEDED, successfulAttemptID=attempt_1463524602570_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524602778, Process_Duration=3, Process_End_Time=1463524602781, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:42,785 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524602570_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:42,785 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0] : 1
2016-05-17 22:36:42,785 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0]
2016-05-17 22:36:42,786 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:42,786 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524602778, Process_Duration=3, Process_End_Time=1463524602781, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:42,786 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602570_0001_1][Event:VERTEX_FINISHED]: vertexName=C135D425603B47868031903B025259F0, vertexId=vertex_1463524602570_0001_1_00, initRequestedTime=1463524602734, initedTime=1463524602744, startRequestedTime=1463524602745, startedTime=1463524602745, finishTime=1463524602785, timeTaken=40, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524602778, Process_Duration=3, Process_End_Time=1463524602781, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524602754, firstTasksToStart=[ task_1463524602570_0001_1_00_000000 ], lastTaskFinishTime=1463524602783, lastTasksToFinish=[ task_1463524602570_0001_1_00_000000 ], minTaskDuration=29, maxTaskDuration=29, avgTaskDuration=29.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524602570_0001_1_00_000000 ], longestDurationTasks=[ task_1463524602570_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:42,786 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:42,786 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:42,786 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524602570_0001_00_000001
2016-05-17 22:36:42,786 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:42,787 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524602570_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:42,787 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602570_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524602570_0001_00_000001, stoppedTime=1463524602787, exitStatus=0
2016-05-17 22:36:42,787 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:42,787 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524602570_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:42,787 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:42,788 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524602570_0001_1
2016-05-17 22:36:42,788 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602570_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524602570_0001_1
2016-05-17 22:36:42,788 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 51C9F67265B5496DB4C324C2B3BBC0C6 for vertex: vertex_1463524602570_0001_1_00 [C135D425603B47868031903B025259F0]
2016-05-17 22:36:42,788 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524602570_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:42,789 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524602570_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:42,789 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524602570_0001_00_000001, Exiting
2016-05-17 22:36:42,789 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524602570_0001_00_000001
2016-05-17 22:36:42,789 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524602570_0001_00_000001
2016-05-17 22:36:42,791 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524602570_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:42,791 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524602570_0001_00_000001 completed successfully
2016-05-17 22:36:42,797 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524602570_0001_1, queueSize=0
2016-05-17 22:36:42,797 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:42,798 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524602570_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524602570_0001_1, startTime=1463524602734, finishTime=1463524602796, timeTaken=62, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524602778, Process_Duration=3, Process_End_Time=1463524602781, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:42,798 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524602570_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:42,798 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524602570_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:42,799 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524602570_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:42,799 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524602570_0001_00_000001
2016-05-17 22:36:42 Completed Dag: dag_1463524602570_0001_1
2016-05-17 22:36:42,800 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524602570_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:42,800 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524602570_0001_1_post
2016-05-17 22:36:42,800 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:42,800 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:42,805 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ssemblyplatform/0_ARGS_-1, applicationId=application_1463524602570_0001
2016-05-17 22:36:42,806 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:42,806 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:42,806 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:42,806 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:42,806 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:42,807 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:42,807 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:42,807 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:42,807 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:42,807 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:42,808 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:42,808 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:42,808 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:42,812 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/82E52E9B069647A19266C86D6C8E8004/E066F5946D4A438BA37C7CB9F957FA85/.tez/application_1463524602570_0001
2016-05-17 22:36:42,812 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_ARGS_-1/_temporary
2016-05-17 22:36:42,813 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.264
2016-05-17 22:36:42,876 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_ARGS_-1
2016-05-17 22:36:42,879 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:42,879 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:42,897 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:42,898 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:42,923 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-17 22:36:42,926 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.006
2016-05-17 22:36:42,926 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:42,926 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:42,926 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_ARGS_'field'
2016-05-17 22:36:42,948 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:42,948 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:42,983 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 22:36:42,988 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 22:36:42,988 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:42,989 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:42,989 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:43,002 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:43,003 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_ARGS_RESULTS
2016-05-17 22:36:43,012 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:43,012 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:43,012 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_ARGS_RESULTS"]
2016-05-17 22:36:43,013 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:43,013 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:43,013 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:43,014 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...lyplatform/0_ARGS_RESULTS
2016-05-17 22:36:43,014 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:43,025 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:43,030 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:43,030 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:43,030 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:43,039 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7A7FF294354D4C61B6E3776DB5677835/E6828168BF50415480F9B42A2F4501BB/.tez/application_1463524603030_0001 doesn't exist and is created
2016-05-17 22:36:43,054 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:43,055 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:43,055 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7A7FF294354D4C61B6E3776DB5677835/E6828168BF50415480F9B42A2F4501BB/.tez/application_1463524603030_0001
2016-05-17 22:36:43,057 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524603030_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:43,067 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:43,069 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:43,070 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:43,070 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:43,070 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7A7FF294354D4C61B6E3776DB5677835/E6828168BF50415480F9B42A2F4501BB/.tez/application_1463524603030_0001/localmode-log-dir
2016-05-17 22:36:43,071 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7A7FF294354D4C61B6E3776DB5677835/E6828168BF50415480F9B42A2F4501BB/.tez/application_1463524603030_0001/localmode-log-dir/history.txt.appattempt_1463524603030_0001_000000, maxErrors=10
2016-05-17 22:36:43,071 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:43,071 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524603030_0001_000000, appSubmitTime=1463524603057, launchTime=1463524603057
2016-05-17 22:36:43,076 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:38766
2016-05-17 22:36:43,078 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:43,078 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:43,080 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:43,081 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:43,082 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524603030_0001_000000, startTime=1463524603082
2016-05-17 22:36:43,083 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:43,083 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:43,083 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:43,155 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:43,155 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:43,155 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/0_ARGS_RESULTS, applicationId=application_1463524603030_0001, dagName=[7A7FF294354D4C61B6E3776DB5677835/E6828168BF50415480F9B42A2F4501BB] each/(1/1) ...lyplatform/0_ARGS_RESULTS
2016-05-17 22:36:43,169 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [7A7FF294354D4C61B6E3776DB5677835/E6828168BF50415480F9B42A2F4501BB] each/(1/1) ...lyplatform/0_ARGS_RESULTS
2016-05-17 22:36:43,170 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524603030_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7A7FF294354D4C61B6E3776DB5677835/E6828168BF50415480F9B42A2F4501BB/.tez/application_1463524603030_0001/localmode-log-dir/dag_1463524603030_0001_1.dot
2016-05-17 22:36:43,170 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7A7FF294354D4C61B6E3776DB5677835/E6828168BF50415480F9B42A2F4501BB/.tez/application_1463524603030_0001/localmode-log-dir/dag_1463524603030_0001_1-tez-dag.pb.txt
2016-05-17 22:36:43,174 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524603030_0001_1
2016-05-17 22:36:43,174 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:43,174 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [7A7FF294354D4C61B6E3776DB5677835/E6828168BF50415480F9B42A2F4501BB] each/(1/1) ...lyplatform/0_ARGS_RESULTS
2016-05-17 22:36:43 Running Dag: dag_1463524603030_0001_1
2016-05-17 22:36:43,184 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603030_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524603030_0001_1, submitTime=1463524603169
2016-05-17 22:36:43,184 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:43,185 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F0AB90EFDB68460EA273A66F6E0F58C3
2016-05-17 22:36:43,185 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:43,185 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603030_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524603030_0001_1, initTime=1463524603184
2016-05-17 22:36:43,185 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:43,186 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524603030_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:43,186 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603030_0001_1][Event:DAG_STARTED]: dagID=dag_1463524603030_0001_1, startTime=1463524603186
2016-05-17 22:36:43,186 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:43,186 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:43,186 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524603030_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:43,187 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F0AB90EFDB68460EA273A66F6E0F58C3 : {0363B24E106040E093FF4B95703B6EDD={InputName=0363B24E106040E093FF4B95703B6EDD}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:43,187 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 0363B24E106040E093FF4B95703B6EDD, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:43,187 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3]
2016-05-17 22:36:43,187 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3]
2016-05-17 22:36:43,187 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3]
2016-05-17 22:36:43,187 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/0_ARGS_RESULTS, applicationId=application_1463524603030_0001, dagName=[7A7FF294354D4C61B6E3776DB5677835/E6828168BF50415480F9B42A2F4501BB] each/(1/1) ...lyplatform/0_ARGS_RESULTS
2016-05-17 22:36:43,187 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3]. Starting root input initializers: 1
2016-05-17 22:36:43,188 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:43,188 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 0363B24E106040E093FF4B95703B6EDD on vertex vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3]
2016-05-17 22:36:43,190 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:43,190 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 0363B24E106040E093FF4B95703B6EDD asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:43,190 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:43,190 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:43,192 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:43,192 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 0363B24E106040E093FF4B95703B6EDD on vertex vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3]
2016-05-17 22:36:43,192 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:43,192 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {0363B24E106040E093FF4B95703B6EDD=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:43,192 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3] parallelism set to 1
2016-05-17 22:36:43,192 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:43,193 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603030_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524603030_0001_1_00, eventCount=1
2016-05-17 22:36:43,193 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3]
2016-05-17 22:36:43,193 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=356BCDBA7B26466C9525AE606525030A, vertexId=vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:43,193 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=356BCDBA7B26466C9525AE606525030A, vertexId=vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3]
2016-05-17 22:36:43,194 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=356BCDBA7B26466C9525AE606525030A, vertexName=F0AB90EFDB68460EA273A66F6E0F58C3, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:43,194 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=356BCDBA7B26466C9525AE606525030A, vertexId=vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3]
2016-05-17 22:36:43,195 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:43,195 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603030_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F0AB90EFDB68460EA273A66F6E0F58C3, vertexId=vertex_1463524603030_0001_1_00, initRequestedTime=1463524603187, initedTime=1463524603195, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:43,195 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3] with distanceFromRoot: 0
2016-05-17 22:36:43,195 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:43,195 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F0AB90EFDB68460EA273A66F6E0F58C3
2016-05-17 22:36:43,196 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3]
2016-05-17 22:36:43,196 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:43,196 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603030_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524603030_0001_1_00, startRequestedTime=1463524603195, startedTime=1463524603195
2016-05-17 22:36:43,196 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:43,197 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:43,197 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603030_0001_1][Event:TASK_STARTED]: vertexName=F0AB90EFDB68460EA273A66F6E0F58C3, taskId=task_1463524603030_0001_1_00_000000, scheduledTime=1463524603197, launchTime=1463524603197
2016-05-17 22:36:43,197 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524603030_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:43,197 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3]
2016-05-17 22:36:43,197 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7A7FF294354D4C61B6E3776DB5677835/E6828168BF50415480F9B42A2F4501BB] each/(1/1) ...lyplatform/0_ARGS_RESULTS, VertexName: F0AB90EFDB68460EA273A66F6E0F58C3, VertexParallelism: 1, TaskAttemptID:attempt_1463524603030_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0363B24E106040E093FF4B95703B6EDD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=356BCDBA7B26466C9525AE606525030A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:43,198 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524603030_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:43,198 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:43,198 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:43,199 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:43,199 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:43,199 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524603030_0001_00_000001
2016-05-17 22:36:43,199 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524603030_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:43,200 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524603030_0001_00_000001
2016-05-17 22:36:43,201 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:43,201 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524603030_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:43,201 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:43,202 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603030_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524603030_0001_00_000001, launchTime=1463524603201
2016-05-17 22:36:43,202 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524603030_0001_00_000001
2016-05-17 22:36:43,203 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524603030_0001_1_00_000000_0] to container: [container_1463524603030_0001_00_000001]
2016-05-17 22:36:43,203 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524603030_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:43,203 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524603030_0001_00_000001 given task: attempt_1463524603030_0001_1_00_000000_0
2016-05-17 22:36:43,203 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524603030_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524603030_0001_1_00_000000_0
2016-05-17 22:36:43,203 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524603030_0001_1_00_000000_0] started. Is using containerId: [container_1463524603030_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:43,203 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:43 Starting to run new task attempt: attempt_1463524603030_0001_1_00_000000_0
2016-05-17 22:36:43,203 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603030_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F0AB90EFDB68460EA273A66F6E0F58C3, taskAttemptId=attempt_1463524603030_0001_1_00_000000_0, startTime=1463524603203, containerId=container_1463524603030_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524603030_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:43,204 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524603030_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:43,204 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524603030_0001_1_00_000000_0
2016-05-17 22:36:43,204 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:43,205 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524603030_0001, with dag id: dag_1463524603030_0001_1
2016-05-17 22:36:43,205 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:43,204 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524603030_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:43,205 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:43,205 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:43,206 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:43,206 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:43,206 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7A7FF294354D4C61B6E3776DB5677835/E6828168BF50415480F9B42A2F4501BB] each/(1/1) ...lyplatform/0_ARGS_RESULTS, VertexName: F0AB90EFDB68460EA273A66F6E0F58C3, VertexParallelism: 1, TaskAttemptID:attempt_1463524603030_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0363B24E106040E093FF4B95703B6EDD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=356BCDBA7B26466C9525AE606525030A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:43,206 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:43,208 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524603030_0001_1_00_000000_0
2016-05-17 22:36:43,208 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:43,209 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:43,209 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524603030_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:43,209 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=356BCDBA7B26466C9525AE606525030A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:43,210 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:43,210 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 356BCDBA7B26466C9525AE606525030A
2016-05-17 22:36:43,211 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:43,211 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 356BCDBA7B26466C9525AE606525030A
2016-05-17 22:36:43,210 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=0363B24E106040E093FF4B95703B6EDD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:43,209 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:43,214 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:43,216 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:43,216 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 0363B24E106040E093FF4B95703B6EDD
2016-05-17 22:36:43,216 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 0363B24E106040E093FF4B95703B6EDD
2016-05-17 22:36:43,218 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:43,218 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 0363B24E106040E093FF4B95703B6EDD
2016-05-17 22:36:43,219 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 0363B24E106040E093FF4B95703B6EDD
2016-05-17 22:36:43,222 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:43,222 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 356BCDBA7B26466C9525AE606525030A
2016-05-17 22:36:43,224 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F0AB90EFDB68460EA273A66F6E0F58C3, ordinal: 0
2016-05-17 22:36:43,224 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F0AB90EFDB68460EA273A66F6E0F58C3, mem on start (mb), free: 176, total: 355, max: 672
2016-05-17 22:36:43,224 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:43,225 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:43,225 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:43,225 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:43,225 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:43,225 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:43,225 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:43,225 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:43,225 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:43,225 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:43,226 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:43,226 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 356BCDBA7B26466C9525AE606525030A, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:43,226 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 0363B24E106040E093FF4B95703B6EDD, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:43,226 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:43,226 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 0363B24E106040E093FF4B95703B6EDD
2016-05-17 22:36:43,226 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:43,226 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 0363B24E106040E093FF4B95703B6EDD
2016-05-17 22:36:43,226 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 0363B24E106040E093FF4B95703B6EDD being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:43,227 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:43,227 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:43,227 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:43,227 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524603030_0001_1_00_000000_0
2016-05-17 22:36:43,229 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:43,229 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:43,230 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 0363B24E106040E093FF4B95703B6EDD
2016-05-17 22:36:43,230 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@56ba837c
2016-05-17 22:36:43,230 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_ARGS_RESULTS"], id: 356BCDBA7B26466C9525AE606525030A
2016-05-17 22:36:43,231 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_ARGS_RESULTS"]
2016-05-17 22:36:43,231 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:43,231 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:43,232 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@56ba837c
2016-05-17 22:36:43,232 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:43,232 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:43,232 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F0AB90EFDB68460EA273A66F6E0F58C3, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:43,235 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:43,235 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:43,236 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524603030_0001_1_00_000000_0
2016-05-17 22:36:43,237 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524603030_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:43,237 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524603030_0001_1_00_000000_0
2016-05-17 22:36:43,237 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F0AB90EFDB68460EA273A66F6E0F58C3, mem on close (mb), free: 175, total: 355, max: 672
2016-05-17 22:36:43 Completed running task attempt: attempt_1463524603030_0001_1_00_000000_0
2016-05-17 22:36:43,238 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524603030_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:43,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524603232, Process_Duration=5, Process_End_Time=1463524603237, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:43,240 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603030_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F0AB90EFDB68460EA273A66F6E0F58C3, taskAttemptId=attempt_1463524603030_0001_1_00_000000_0, startTime=1463524603203, finishTime=1463524603239, timeTaken=36, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524603232, Process_Duration=5, Process_End_Time=1463524603237, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:43,240 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:43,240 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:43,239 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:43,240 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524603030_0001_00_000001
2016-05-17 22:36:43,240 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524603030_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:43,241 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524603030_0001_1_00_000000_0
2016-05-17 22:36:43,241 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603030_0001_1][Event:TASK_FINISHED]: vertexName=F0AB90EFDB68460EA273A66F6E0F58C3, taskId=task_1463524603030_0001_1_00_000000, startTime=1463524603203, finishTime=1463524603241, timeTaken=38, status=SUCCEEDED, successfulAttemptID=attempt_1463524603030_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524603232, Process_Duration=5, Process_End_Time=1463524603237, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:43,241 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524603030_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:43,241 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3] : 1
2016-05-17 22:36:43,242 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3]
2016-05-17 22:36:43,242 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603030_0001_1][Event:VERTEX_FINISHED]: vertexName=F0AB90EFDB68460EA273A66F6E0F58C3, vertexId=vertex_1463524603030_0001_1_00, initRequestedTime=1463524603187, initedTime=1463524603195, startRequestedTime=1463524603195, startedTime=1463524603195, finishTime=1463524603242, timeTaken=47, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524603232, Process_Duration=5, Process_End_Time=1463524603237, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524603203, firstTasksToStart=[ task_1463524603030_0001_1_00_000000 ], lastTaskFinishTime=1463524603239, lastTasksToFinish=[ task_1463524603030_0001_1_00_000000 ], minTaskDuration=36, maxTaskDuration=36, avgTaskDuration=36.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524603030_0001_1_00_000000 ], longestDurationTasks=[ task_1463524603030_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:43,242 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:43,242 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:43,243 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524603030_0001_1
2016-05-17 22:36:43,243 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603030_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524603030_0001_1
2016-05-17 22:36:43,243 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 356BCDBA7B26466C9525AE606525030A for vertex: vertex_1463524603030_0001_1_00 [F0AB90EFDB68460EA273A66F6E0F58C3]
2016-05-17 22:36:43,244 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:43,245 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:43,245 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:43,252 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524603030_0001_1, queueSize=0
2016-05-17 22:36:43,252 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:43,252 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603030_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524603030_0001_1, startTime=1463524603186, finishTime=1463524603251, timeTaken=65, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524603232, Process_Duration=5, Process_End_Time=1463524603237, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:43,252 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524603030_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:43,252 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524603030_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:43,253 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524603030_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:43,254 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603030_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524603030_0001_00_000001, stoppedTime=1463524603254, exitStatus=0
2016-05-17 22:36:43,254 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:43,255 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524603030_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:43 Completed Dag: dag_1463524603030_0001_1
2016-05-17 22:36:43,258 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524603030_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:43,258 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524603030_0001_1_post
2016-05-17 22:36:43,258 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:43,259 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:43,261 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/0_ARGS_RESULTS, applicationId=application_1463524603030_0001
2016-05-17 22:36:43,261 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524603030_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:36:43,262 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524603030_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:36:43,262 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:43,262 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:43,262 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:43,263 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:43,263 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:43,263 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:43,263 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:43,264 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:43,264 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:43,264 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:43,265 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:43,265 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:43,265 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:43,265 INFO task.TezChild (TezChild.java:run(198)) - Interrupted while waiting for new work for container container_1463524603030_0001_00_000001
2016-05-17 22:36:43,266 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524603030_0001_00_000001
2016-05-17 22:36:43,266 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524603030_0001_00_000001
2016-05-17 22:36:43,271 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7A7FF294354D4C61B6E3776DB5677835/E6828168BF50415480F9B42A2F4501BB/.tez/application_1463524603030_0001
2016-05-17 22:36:43,272 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_ARGS_RESULTS/_temporary
2016-05-17 22:36:43,272 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.260
2016-05-17 22:36:43,335 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_ARGS_RESULTS
2016-05-17 22:36:43,356 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:43,356 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:43,382 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-05-17 22:36:43,385 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.006
2016-05-17 22:36:43,385 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:43,385 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:43,385 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:43,386 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:43,386 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_ARGS_ALL
2016-05-17 22:36:43,414 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:43,415 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:43,443 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-17 22:36:43,451 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-17 22:36:43,451 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:43,451 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:43,451 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:43,464 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:43,464 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_ARGS_REPLACE
2016-05-17 22:36:43,472 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:43,472 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:43,472 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_ARGS_REPLACE"]
2016-05-17 22:36:43,472 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:43,473 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:43,473 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:43,474 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...lyplatform/0_ARGS_REPLACE
2016-05-17 22:36:43,474 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:43,485 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:43,489 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:43,490 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:43,490 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:43,500 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CDEEF33A63124FC881BFCA8126EA0BB9/F8FDC81CAE3C4EB4B9A99B13EBDE69D0/.tez/application_1463524603490_0001 doesn't exist and is created
2016-05-17 22:36:43,516 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:43,516 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:43,516 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CDEEF33A63124FC881BFCA8126EA0BB9/F8FDC81CAE3C4EB4B9A99B13EBDE69D0/.tez/application_1463524603490_0001
2016-05-17 22:36:43,521 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524603490_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:43,530 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:43,531 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:43,532 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:43,532 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:43,532 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CDEEF33A63124FC881BFCA8126EA0BB9/F8FDC81CAE3C4EB4B9A99B13EBDE69D0/.tez/application_1463524603490_0001/localmode-log-dir
2016-05-17 22:36:43,532 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CDEEF33A63124FC881BFCA8126EA0BB9/F8FDC81CAE3C4EB4B9A99B13EBDE69D0/.tez/application_1463524603490_0001/localmode-log-dir/history.txt.appattempt_1463524603490_0001_000000, maxErrors=10
2016-05-17 22:36:43,533 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:43,533 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524603490_0001_000000, appSubmitTime=1463524603520, launchTime=1463524603521
2016-05-17 22:36:43,535 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:43,537 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:43,537 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:43,538 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:43,539 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:55464
2016-05-17 22:36:43,540 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:43,542 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524603490_0001_000000, startTime=1463524603542
2016-05-17 22:36:43,542 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:43,542 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:43,616 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:43,616 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:43,616 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/0_ARGS_REPLACE, applicationId=application_1463524603490_0001, dagName=[CDEEF33A63124FC881BFCA8126EA0BB9/F8FDC81CAE3C4EB4B9A99B13EBDE69D0] each/(1/1) ...lyplatform/0_ARGS_REPLACE
2016-05-17 22:36:43,643 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [CDEEF33A63124FC881BFCA8126EA0BB9/F8FDC81CAE3C4EB4B9A99B13EBDE69D0] each/(1/1) ...lyplatform/0_ARGS_REPLACE
2016-05-17 22:36:43,644 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524603490_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CDEEF33A63124FC881BFCA8126EA0BB9/F8FDC81CAE3C4EB4B9A99B13EBDE69D0/.tez/application_1463524603490_0001/localmode-log-dir/dag_1463524603490_0001_1.dot
2016-05-17 22:36:43,644 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CDEEF33A63124FC881BFCA8126EA0BB9/F8FDC81CAE3C4EB4B9A99B13EBDE69D0/.tez/application_1463524603490_0001/localmode-log-dir/dag_1463524603490_0001_1-tez-dag.pb.txt
2016-05-17 22:36:43,648 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524603490_0001_1
2016-05-17 22:36:43,648 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:43,648 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [CDEEF33A63124FC881BFCA8126EA0BB9/F8FDC81CAE3C4EB4B9A99B13EBDE69D0] each/(1/1) ...lyplatform/0_ARGS_REPLACE
2016-05-17 22:36:43 Running Dag: dag_1463524603490_0001_1
2016-05-17 22:36:43,659 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:43,659 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603490_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524603490_0001_1, submitTime=1463524603643
2016-05-17 22:36:43,660 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F8771CD7C7C84E17AF4618C4323E2990
2016-05-17 22:36:43,660 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:43,660 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:43,661 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603490_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524603490_0001_1, initTime=1463524603659
2016-05-17 22:36:43,661 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524603490_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:43,661 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603490_0001_1][Event:DAG_STARTED]: dagID=dag_1463524603490_0001_1, startTime=1463524603661
2016-05-17 22:36:43,661 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:43,661 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524603490_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:43,661 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F8771CD7C7C84E17AF4618C4323E2990 : {1B1F327E44AA4DC1A0CCBA93FC1C4C03={InputName=1B1F327E44AA4DC1A0CCBA93FC1C4C03}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:43,661 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 1B1F327E44AA4DC1A0CCBA93FC1C4C03, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:43,662 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990]
2016-05-17 22:36:43,662 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990]
2016-05-17 22:36:43,662 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990]
2016-05-17 22:36:43,662 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990]. Starting root input initializers: 1
2016-05-17 22:36:43,662 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:43,662 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:43,662 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 1B1F327E44AA4DC1A0CCBA93FC1C4C03 on vertex vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990]
2016-05-17 22:36:43,663 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/0_ARGS_REPLACE, applicationId=application_1463524603490_0001, dagName=[CDEEF33A63124FC881BFCA8126EA0BB9/F8FDC81CAE3C4EB4B9A99B13EBDE69D0] each/(1/1) ...lyplatform/0_ARGS_REPLACE
2016-05-17 22:36:43,664 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:43,664 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 1B1F327E44AA4DC1A0CCBA93FC1C4C03 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:43,665 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:43,665 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:43,666 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:43,666 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 1B1F327E44AA4DC1A0CCBA93FC1C4C03 on vertex vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990]
2016-05-17 22:36:43,666 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:43,667 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {1B1F327E44AA4DC1A0CCBA93FC1C4C03=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:43,667 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990] parallelism set to 1
2016-05-17 22:36:43,667 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:43,667 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603490_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524603490_0001_1_00, eventCount=1
2016-05-17 22:36:43,667 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990]
2016-05-17 22:36:43,667 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A447DD0A2B3C483485F0CF826DD4C6B2, vertexId=vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:43,668 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A447DD0A2B3C483485F0CF826DD4C6B2, vertexId=vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990]
2016-05-17 22:36:43,669 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A447DD0A2B3C483485F0CF826DD4C6B2, vertexName=F8771CD7C7C84E17AF4618C4323E2990, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:43,669 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A447DD0A2B3C483485F0CF826DD4C6B2, vertexId=vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990]
2016-05-17 22:36:43,671 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603490_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F8771CD7C7C84E17AF4618C4323E2990, vertexId=vertex_1463524603490_0001_1_00, initRequestedTime=1463524603661, initedTime=1463524603670, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:43,671 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:43,671 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990] with distanceFromRoot: 0
2016-05-17 22:36:43,671 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:43,671 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F8771CD7C7C84E17AF4618C4323E2990
2016-05-17 22:36:43,671 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990]
2016-05-17 22:36:43,671 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603490_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524603490_0001_1_00, startRequestedTime=1463524603671, startedTime=1463524603671
2016-05-17 22:36:43,671 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:43,672 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:43,672 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:43,672 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603490_0001_1][Event:TASK_STARTED]: vertexName=F8771CD7C7C84E17AF4618C4323E2990, taskId=task_1463524603490_0001_1_00_000000, scheduledTime=1463524603672, launchTime=1463524603672
2016-05-17 22:36:43,672 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524603490_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:43,673 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990]
2016-05-17 22:36:43,673 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [CDEEF33A63124FC881BFCA8126EA0BB9/F8FDC81CAE3C4EB4B9A99B13EBDE69D0] each/(1/1) ...lyplatform/0_ARGS_REPLACE, VertexName: F8771CD7C7C84E17AF4618C4323E2990, VertexParallelism: 1, TaskAttemptID:attempt_1463524603490_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=1B1F327E44AA4DC1A0CCBA93FC1C4C03, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A447DD0A2B3C483485F0CF826DD4C6B2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:43,673 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524603490_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:43,673 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:43,674 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:43,674 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:43,675 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:43,675 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524603490_0001_00_000001
2016-05-17 22:36:43,675 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524603490_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:43,677 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524603490_0001_00_000001
2016-05-17 22:36:43,677 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:43,677 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524603490_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:43,678 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:43,678 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603490_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524603490_0001_00_000001, launchTime=1463524603678
2016-05-17 22:36:43,679 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524603490_0001_00_000001
2016-05-17 22:36:43,680 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524603490_0001_1_00_000000_0] to container: [container_1463524603490_0001_00_000001]
2016-05-17 22:36:43,680 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524603490_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:43,680 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524603490_0001_00_000001 given task: attempt_1463524603490_0001_1_00_000000_0
2016-05-17 22:36:43,680 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524603490_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524603490_0001_1_00_000000_0
2016-05-17 22:36:43,680 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524603490_0001_1_00_000000_0] started. Is using containerId: [container_1463524603490_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:43,680 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603490_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F8771CD7C7C84E17AF4618C4323E2990, taskAttemptId=attempt_1463524603490_0001_1_00_000000_0, startTime=1463524603680, containerId=container_1463524603490_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524603490_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:43,681 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524603490_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:43 Starting to run new task attempt: attempt_1463524603490_0001_1_00_000000_0
2016-05-17 22:36:43,681 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524603490_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:43,680 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:43,681 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524603490_0001_1_00_000000_0
2016-05-17 22:36:43,681 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:43,682 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:43,682 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:43,682 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:43,682 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:43,682 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524603490_0001, with dag id: dag_1463524603490_0001_1
2016-05-17 22:36:43,682 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:43,683 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [CDEEF33A63124FC881BFCA8126EA0BB9/F8FDC81CAE3C4EB4B9A99B13EBDE69D0] each/(1/1) ...lyplatform/0_ARGS_REPLACE, VertexName: F8771CD7C7C84E17AF4618C4323E2990, VertexParallelism: 1, TaskAttemptID:attempt_1463524603490_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=1B1F327E44AA4DC1A0CCBA93FC1C4C03, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A447DD0A2B3C483485F0CF826DD4C6B2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:43,683 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:43,683 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524603490_0001_1_00_000000_0
2016-05-17 22:36:43,684 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:43,684 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:43,684 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:43,685 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=1B1F327E44AA4DC1A0CCBA93FC1C4C03, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:43,687 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A447DD0A2B3C483485F0CF826DD4C6B2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:43,687 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:43,688 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:43,688 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:43,688 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 1B1F327E44AA4DC1A0CCBA93FC1C4C03
2016-05-17 22:36:43,687 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524603490_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:43,689 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 1B1F327E44AA4DC1A0CCBA93FC1C4C03
2016-05-17 22:36:43,689 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A447DD0A2B3C483485F0CF826DD4C6B2
2016-05-17 22:36:43,689 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:43,689 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A447DD0A2B3C483485F0CF826DD4C6B2
2016-05-17 22:36:43,692 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:43,692 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 1B1F327E44AA4DC1A0CCBA93FC1C4C03
2016-05-17 22:36:43,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 1B1F327E44AA4DC1A0CCBA93FC1C4C03
2016-05-17 22:36:43,695 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F8771CD7C7C84E17AF4618C4323E2990, ordinal: 0
2016-05-17 22:36:43,695 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F8771CD7C7C84E17AF4618C4323E2990, mem on start (mb), free: 269, total: 350, max: 672
2016-05-17 22:36:43,696 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:43,696 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:43,696 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:43,699 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:43,699 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A447DD0A2B3C483485F0CF826DD4C6B2
2016-05-17 22:36:43,699 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:43,699 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:43,699 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:43,700 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:43,700 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:43,700 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:43,700 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:43,700 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:43,700 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 1B1F327E44AA4DC1A0CCBA93FC1C4C03, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:43,700 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A447DD0A2B3C483485F0CF826DD4C6B2, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:43,700 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:43,700 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 1B1F327E44AA4DC1A0CCBA93FC1C4C03 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:43,700 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 1B1F327E44AA4DC1A0CCBA93FC1C4C03
2016-05-17 22:36:43,700 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:43,701 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:43,700 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:43,701 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 1B1F327E44AA4DC1A0CCBA93FC1C4C03
2016-05-17 22:36:43,702 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:43,702 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524603490_0001_1_00_000000_0
2016-05-17 22:36:43,702 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:43,703 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@65c36ecc
2016-05-17 22:36:43,703 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@65c36ecc
2016-05-17 22:36:43,704 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:43,704 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:43,704 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:43,705 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 1B1F327E44AA4DC1A0CCBA93FC1C4C03
2016-05-17 22:36:43,705 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_ARGS_REPLACE"], id: A447DD0A2B3C483485F0CF826DD4C6B2
2016-05-17 22:36:43,705 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_ARGS_REPLACE"]
2016-05-17 22:36:43,705 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:43,705 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:43,705 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F8771CD7C7C84E17AF4618C4323E2990, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:43,708 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:43,708 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:43,708 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524603490_0001_1_00_000000_0
2016-05-17 22:36:43,708 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524603490_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:43,709 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524603490_0001_1_00_000000_0
2016-05-17 22:36:43,709 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F8771CD7C7C84E17AF4618C4323E2990, mem on close (mb), free: 268, total: 350, max: 672
2016-05-17 22:36:43 Completed running task attempt: attempt_1463524603490_0001_1_00_000000_0
2016-05-17 22:36:43,710 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524603490_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:43,710 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=143, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=367525888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524603705, Process_Duration=4, Process_End_Time=1463524603709, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:43,710 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:43,711 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603490_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F8771CD7C7C84E17AF4618C4323E2990, taskAttemptId=attempt_1463524603490_0001_1_00_000000_0, startTime=1463524603680, finishTime=1463524603710, timeTaken=30, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=143, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=367525888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524603705, Process_Duration=4, Process_End_Time=1463524603709, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:43,711 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524603490_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:43,711 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524603490_0001_1_00_000000_0
2016-05-17 22:36:43,711 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:43,711 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:43,712 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:43,712 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:43,712 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524603490_0001_00_000001
2016-05-17 22:36:43,712 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603490_0001_1][Event:TASK_FINISHED]: vertexName=F8771CD7C7C84E17AF4618C4323E2990, taskId=task_1463524603490_0001_1_00_000000, startTime=1463524603680, finishTime=1463524603711, timeTaken=31, status=SUCCEEDED, successfulAttemptID=attempt_1463524603490_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=143, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=367525888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524603705, Process_Duration=4, Process_End_Time=1463524603709, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:43,712 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524603490_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:43,713 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990] : 1
2016-05-17 22:36:43,713 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990]
2016-05-17 22:36:43,713 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:43,714 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603490_0001_1][Event:VERTEX_FINISHED]: vertexName=F8771CD7C7C84E17AF4618C4323E2990, vertexId=vertex_1463524603490_0001_1_00, initRequestedTime=1463524603661, initedTime=1463524603670, startRequestedTime=1463524603671, startedTime=1463524603671, finishTime=1463524603713, timeTaken=42, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=143, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=367525888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524603705, Process_Duration=4, Process_End_Time=1463524603709, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524603680, firstTasksToStart=[ task_1463524603490_0001_1_00_000000 ], lastTaskFinishTime=1463524603710, lastTasksToFinish=[ task_1463524603490_0001_1_00_000000 ], minTaskDuration=30, maxTaskDuration=30, avgTaskDuration=30.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524603490_0001_1_00_000000 ], longestDurationTasks=[ task_1463524603490_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:43,714 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:43,714 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524603490_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:43,714 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603490_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524603490_0001_00_000001, stoppedTime=1463524603714, exitStatus=0
2016-05-17 22:36:43,714 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524603490_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:43,714 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:43,714 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:43,714 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524603490_0001_1
2016-05-17 22:36:43,714 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603490_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524603490_0001_1
2016-05-17 22:36:43,714 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: A447DD0A2B3C483485F0CF826DD4C6B2 for vertex: vertex_1463524603490_0001_1_00 [F8771CD7C7C84E17AF4618C4323E2990]
2016-05-17 22:36:43,721 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524603490_0001_1, queueSize=0
2016-05-17 22:36:43,722 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:43,722 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603490_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524603490_0001_1, startTime=1463524603661, finishTime=1463524603721, timeTaken=60, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=143, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=367525888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524603705, Process_Duration=4, Process_End_Time=1463524603709, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:43,722 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524603490_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:43,722 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524603490_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:43,722 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524603490_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:36:43,722 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524603490_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:43 Completed Dag: dag_1463524603490_0001_1
2016-05-17 22:36:43,722 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524603490_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:43,723 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524603490_0001_1_post
2016-05-17 22:36:43,723 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:43,723 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:43,722 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524603490_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:43,723 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524603490_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:36:43,723 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524603490_0001_00_000001, Exiting
2016-05-17 22:36:43,723 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524603490_0001_00_000001
2016-05-17 22:36:43,723 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524603490_0001_00_000001
2016-05-17 22:36:43,725 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524603490_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:43,725 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524603490_0001_00_000001 completed successfully
2016-05-17 22:36:43,725 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524603490_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:36:43,731 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/0_ARGS_REPLACE, applicationId=application_1463524603490_0001
2016-05-17 22:36:43,733 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:43,733 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:43,733 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:43,733 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:43,733 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:43,733 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:43,734 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:43,734 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:43,734 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:43,734 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:43,734 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:43,734 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:43,737 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:43,739 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CDEEF33A63124FC881BFCA8126EA0BB9/F8FDC81CAE3C4EB4B9A99B13EBDE69D0/.tez/application_1463524603490_0001
2016-05-17 22:36:43,740 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_ARGS_REPLACE/_temporary
2016-05-17 22:36:43,740 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.268
2016-05-17 22:36:43,802 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_ARGS_REPLACE
2016-05-17 22:36:43,812 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:43,812 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:43,822 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:43,822 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:43,855 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 22:36:43,856 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 22:36:43,856 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:43,856 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:43,857 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:43,869 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:43,870 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_ARGS_SWAP
2016-05-17 22:36:43,878 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:43,879 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:43,879 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_ARGS_SWAP"]
2016-05-17 22:36:43,879 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:43,880 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:43,880 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:43,880 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...emblyplatform/0_ARGS_SWAP
2016-05-17 22:36:43,881 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:43,898 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:43,903 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:43,904 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:43,904 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:43,914 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1AD84105BE874DA28DAB39A4CFAD5224/EDBF602E60D84480B5C695655854420E/.tez/application_1463524603903_0001 doesn't exist and is created
2016-05-17 22:36:43,935 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:43,936 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:43,936 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1AD84105BE874DA28DAB39A4CFAD5224/EDBF602E60D84480B5C695655854420E/.tez/application_1463524603903_0001
2016-05-17 22:36:43,939 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524603903_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:43,950 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:43,950 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:43,951 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:43,952 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:43,952 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1AD84105BE874DA28DAB39A4CFAD5224/EDBF602E60D84480B5C695655854420E/.tez/application_1463524603903_0001/localmode-log-dir
2016-05-17 22:36:43,952 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1AD84105BE874DA28DAB39A4CFAD5224/EDBF602E60D84480B5C695655854420E/.tez/application_1463524603903_0001/localmode-log-dir/history.txt.appattempt_1463524603903_0001_000000, maxErrors=10
2016-05-17 22:36:43,952 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:43,952 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524603903_0001_000000, appSubmitTime=1463524603939, launchTime=1463524603939
2016-05-17 22:36:43,958 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:43,959 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:53737
2016-05-17 22:36:43,959 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:43,961 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:43,963 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:43,964 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:43,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524603903_0001_000000, startTime=1463524603966
2016-05-17 22:36:43,966 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:43,966 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:44,036 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:44,036 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:44,036 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...emblyplatform/0_ARGS_SWAP, applicationId=application_1463524603903_0001, dagName=[1AD84105BE874DA28DAB39A4CFAD5224/EDBF602E60D84480B5C695655854420E] each/(1/1) ...emblyplatform/0_ARGS_SWAP
2016-05-17 22:36:44,052 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [1AD84105BE874DA28DAB39A4CFAD5224/EDBF602E60D84480B5C695655854420E] each/(1/1) ...emblyplatform/0_ARGS_SWAP
2016-05-17 22:36:44,053 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524603903_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1AD84105BE874DA28DAB39A4CFAD5224/EDBF602E60D84480B5C695655854420E/.tez/application_1463524603903_0001/localmode-log-dir/dag_1463524603903_0001_1.dot
2016-05-17 22:36:44,053 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1AD84105BE874DA28DAB39A4CFAD5224/EDBF602E60D84480B5C695655854420E/.tez/application_1463524603903_0001/localmode-log-dir/dag_1463524603903_0001_1-tez-dag.pb.txt
2016-05-17 22:36:44,057 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524603903_0001_1
2016-05-17 22:36:44,057 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:44,057 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [1AD84105BE874DA28DAB39A4CFAD5224/EDBF602E60D84480B5C695655854420E] each/(1/1) ...emblyplatform/0_ARGS_SWAP
2016-05-17 22:36:44 Running Dag: dag_1463524603903_0001_1
2016-05-17 22:36:44,068 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:44,068 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603903_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524603903_0001_1, submitTime=1463524604052
2016-05-17 22:36:44,068 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DF4B3B1F55DD422E8F7BD748B04310E7
2016-05-17 22:36:44,069 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:44,069 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:44,069 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603903_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524603903_0001_1, initTime=1463524604068
2016-05-17 22:36:44,069 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524603903_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:44,070 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603903_0001_1][Event:DAG_STARTED]: dagID=dag_1463524603903_0001_1, startTime=1463524604069
2016-05-17 22:36:44,070 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:44,070 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524603903_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:44,070 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: DF4B3B1F55DD422E8F7BD748B04310E7 : {C6C50CFF461349F39F37909AEE9E499D={InputName=C6C50CFF461349F39F37909AEE9E499D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:44,070 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: C6C50CFF461349F39F37909AEE9E499D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:44,070 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...emblyplatform/0_ARGS_SWAP, applicationId=application_1463524603903_0001, dagName=[1AD84105BE874DA28DAB39A4CFAD5224/EDBF602E60D84480B5C695655854420E] each/(1/1) ...emblyplatform/0_ARGS_SWAP
2016-05-17 22:36:44,070 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:44,070 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7]
2016-05-17 22:36:44,070 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7]
2016-05-17 22:36:44,070 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7]
2016-05-17 22:36:44,071 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7]. Starting root input initializers: 1
2016-05-17 22:36:44,071 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:44,071 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: C6C50CFF461349F39F37909AEE9E499D on vertex vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7]
2016-05-17 22:36:44,073 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:44,073 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input C6C50CFF461349F39F37909AEE9E499D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:44,073 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:44,073 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:44,074 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:44,075 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: C6C50CFF461349F39F37909AEE9E499D on vertex vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7]
2016-05-17 22:36:44,075 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:44,075 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {C6C50CFF461349F39F37909AEE9E499D=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:44,075 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7] parallelism set to 1
2016-05-17 22:36:44,075 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:44,075 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603903_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524603903_0001_1_00, eventCount=1
2016-05-17 22:36:44,075 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7]
2016-05-17 22:36:44,076 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=F5A1EA0417C44158BCD50D25E1E962D4, vertexId=vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:44,076 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=F5A1EA0417C44158BCD50D25E1E962D4, vertexId=vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7]
2016-05-17 22:36:44,077 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=F5A1EA0417C44158BCD50D25E1E962D4, vertexName=DF4B3B1F55DD422E8F7BD748B04310E7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:44,077 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=F5A1EA0417C44158BCD50D25E1E962D4, vertexId=vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7]
2016-05-17 22:36:44,078 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603903_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DF4B3B1F55DD422E8F7BD748B04310E7, vertexId=vertex_1463524603903_0001_1_00, initRequestedTime=1463524604070, initedTime=1463524604078, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:44,078 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7] with distanceFromRoot: 0
2016-05-17 22:36:44,078 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:44,078 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in DF4B3B1F55DD422E8F7BD748B04310E7
2016-05-17 22:36:44,078 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:44,078 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7]
2016-05-17 22:36:44,079 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:44,079 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603903_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524603903_0001_1_00, startRequestedTime=1463524604078, startedTime=1463524604078
2016-05-17 22:36:44,079 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:44,079 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:44,080 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603903_0001_1][Event:TASK_STARTED]: vertexName=DF4B3B1F55DD422E8F7BD748B04310E7, taskId=task_1463524603903_0001_1_00_000000, scheduledTime=1463524604079, launchTime=1463524604079
2016-05-17 22:36:44,080 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524603903_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:44,080 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7]
2016-05-17 22:36:44,080 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1AD84105BE874DA28DAB39A4CFAD5224/EDBF602E60D84480B5C695655854420E] each/(1/1) ...emblyplatform/0_ARGS_SWAP, VertexName: DF4B3B1F55DD422E8F7BD748B04310E7, VertexParallelism: 1, TaskAttemptID:attempt_1463524603903_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C6C50CFF461349F39F37909AEE9E499D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F5A1EA0417C44158BCD50D25E1E962D4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:44,081 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524603903_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:44,081 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:44,081 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:44,082 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:44,082 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:44,082 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524603903_0001_00_000001
2016-05-17 22:36:44,083 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524603903_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:44,084 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524603903_0001_00_000001
2016-05-17 22:36:44,084 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:44,085 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524603903_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:44,085 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603903_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524603903_0001_00_000001, launchTime=1463524604085
2016-05-17 22:36:44,085 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:44,086 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524603903_0001_00_000001
2016-05-17 22:36:44,087 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524603903_0001, with dag id: dag_1463524603903_0001_1
2016-05-17 22:36:44,087 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524603903_0001_1_00_000000_0] to container: [container_1463524603903_0001_00_000001]
2016-05-17 22:36:44,087 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524603903_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:44,088 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524603903_0001_00_000001 given task: attempt_1463524603903_0001_1_00_000000_0
2016-05-17 22:36:44,088 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524603903_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524603903_0001_1_00_000000_0
2016-05-17 22:36:44 Starting to run new task attempt: attempt_1463524603903_0001_1_00_000000_0
2016-05-17 22:36:44,088 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524603903_0001_1_00_000000_0] started. Is using containerId: [container_1463524603903_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:44,088 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524603903_0001_1_00_000000_0
2016-05-17 22:36:44,089 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:44,089 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:44,089 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:44,089 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:44,089 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:44,089 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:44,089 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1AD84105BE874DA28DAB39A4CFAD5224/EDBF602E60D84480B5C695655854420E] each/(1/1) ...emblyplatform/0_ARGS_SWAP, VertexName: DF4B3B1F55DD422E8F7BD748B04310E7, VertexParallelism: 1, TaskAttemptID:attempt_1463524603903_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C6C50CFF461349F39F37909AEE9E499D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F5A1EA0417C44158BCD50D25E1E962D4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:44,089 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:44,090 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524603903_0001_1_00_000000_0
2016-05-17 22:36:44,090 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:44,090 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:44,092 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603903_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DF4B3B1F55DD422E8F7BD748B04310E7, taskAttemptId=attempt_1463524603903_0001_1_00_000000_0, startTime=1463524604088, containerId=container_1463524603903_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524603903_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:44,092 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:44,093 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524603903_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:44,093 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524603903_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:44,093 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524603903_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:44,093 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:44,094 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F5A1EA0417C44158BCD50D25E1E962D4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:44,094 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:44,094 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F5A1EA0417C44158BCD50D25E1E962D4
2016-05-17 22:36:44,094 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:44,094 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: F5A1EA0417C44158BCD50D25E1E962D4
2016-05-17 22:36:44,096 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C6C50CFF461349F39F37909AEE9E499D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:44,096 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:44,096 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C6C50CFF461349F39F37909AEE9E499D
2016-05-17 22:36:44,096 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: C6C50CFF461349F39F37909AEE9E499D
2016-05-17 22:36:44,098 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:44,098 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:44,099 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: C6C50CFF461349F39F37909AEE9E499D
2016-05-17 22:36:44,099 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C6C50CFF461349F39F37909AEE9E499D
2016-05-17 22:36:44,105 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:44,105 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F5A1EA0417C44158BCD50D25E1E962D4
2016-05-17 22:36:44,109 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DF4B3B1F55DD422E8F7BD748B04310E7, ordinal: 0
2016-05-17 22:36:44,110 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DF4B3B1F55DD422E8F7BD748B04310E7, mem on start (mb), free: 194, total: 350, max: 672
2016-05-17 22:36:44,110 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:44,110 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:44,110 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:44,111 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:44,111 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:44,111 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:44,112 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:44,112 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:44,112 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:44,113 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:44,113 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:44,114 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F5A1EA0417C44158BCD50D25E1E962D4, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:44,114 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C6C50CFF461349F39F37909AEE9E499D, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:44,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:44,115 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C6C50CFF461349F39F37909AEE9E499D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:44,115 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C6C50CFF461349F39F37909AEE9E499D
2016-05-17 22:36:44,115 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:44,115 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C6C50CFF461349F39F37909AEE9E499D
2016-05-17 22:36:44,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:44,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:44,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:44,117 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524603903_0001_1_00_000000_0
2016-05-17 22:36:44,119 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:44,119 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2d8623c9
2016-05-17 22:36:44,120 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:44,120 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2d8623c9
2016-05-17 22:36:44,120 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:44,121 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:44,121 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: C6C50CFF461349F39F37909AEE9E499D
2016-05-17 22:36:44,121 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_ARGS_SWAP"], id: F5A1EA0417C44158BCD50D25E1E962D4
2016-05-17 22:36:44,121 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_ARGS_SWAP"]
2016-05-17 22:36:44,121 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:44,121 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:44,122 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DF4B3B1F55DD422E8F7BD748B04310E7, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:44,125 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:44,126 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:44,126 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524603903_0001_1_00_000000_0
2016-05-17 22:36:44,126 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524603903_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:44,127 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524603903_0001_1_00_000000_0
2016-05-17 22:36:44,127 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DF4B3B1F55DD422E8F7BD748B04310E7, mem on close (mb), free: 193, total: 350, max: 672
2016-05-17 22:36:44 Completed running task attempt: attempt_1463524603903_0001_1_00_000000_0
2016-05-17 22:36:44,128 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524603903_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:44,129 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=663, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=367525888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524604122, Process_Duration=5, Process_End_Time=1463524604127, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:44,129 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:44,129 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:44,129 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603903_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DF4B3B1F55DD422E8F7BD748B04310E7, taskAttemptId=attempt_1463524603903_0001_1_00_000000_0, startTime=1463524604088, finishTime=1463524604129, timeTaken=41, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=663, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=367525888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524604122, Process_Duration=5, Process_End_Time=1463524604127, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:44,130 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524603903_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:44,130 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524603903_0001_1_00_000000_0
2016-05-17 22:36:44,130 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:44,130 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:44,131 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603903_0001_1][Event:TASK_FINISHED]: vertexName=DF4B3B1F55DD422E8F7BD748B04310E7, taskId=task_1463524603903_0001_1_00_000000, startTime=1463524604088, finishTime=1463524604130, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1463524603903_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=663, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=367525888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524604122, Process_Duration=5, Process_End_Time=1463524604127, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:44,131 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:44,131 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524603903_0001_00_000001
2016-05-17 22:36:44,131 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524603903_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:44,132 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7] : 1
2016-05-17 22:36:44,132 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7]
2016-05-17 22:36:44,133 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:44,133 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603903_0001_1][Event:VERTEX_FINISHED]: vertexName=DF4B3B1F55DD422E8F7BD748B04310E7, vertexId=vertex_1463524603903_0001_1_00, initRequestedTime=1463524604070, initedTime=1463524604078, startRequestedTime=1463524604078, startedTime=1463524604078, finishTime=1463524604132, timeTaken=54, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=663, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=367525888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524604122, Process_Duration=5, Process_End_Time=1463524604127, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524604088, firstTasksToStart=[ task_1463524603903_0001_1_00_000000 ], lastTaskFinishTime=1463524604129, lastTasksToFinish=[ task_1463524603903_0001_1_00_000000 ], minTaskDuration=41, maxTaskDuration=41, avgTaskDuration=41.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524603903_0001_1_00_000000 ], longestDurationTasks=[ task_1463524603903_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:44,133 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:44,133 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524603903_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:44,133 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603903_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524603903_0001_00_000001, stoppedTime=1463524604133, exitStatus=0
2016-05-17 22:36:44,133 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:44,133 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524603903_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:44,134 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:44,134 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524603903_0001_1
2016-05-17 22:36:44,134 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603903_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524603903_0001_1
2016-05-17 22:36:44,134 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: F5A1EA0417C44158BCD50D25E1E962D4 for vertex: vertex_1463524603903_0001_1_00 [DF4B3B1F55DD422E8F7BD748B04310E7]
2016-05-17 22:36:44,141 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524603903_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:44,141 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524603903_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:44,142 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524603903_0001_00_000001, Exiting
2016-05-17 22:36:44,142 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524603903_0001_00_000001
2016-05-17 22:36:44,142 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524603903_0001_00_000001
2016-05-17 22:36:44,144 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524603903_0001_1, queueSize=0
2016-05-17 22:36:44,144 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:44,144 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524603903_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524603903_0001_1, startTime=1463524604069, finishTime=1463524604143, timeTaken=74, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=663, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=367525888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524604122, Process_Duration=5, Process_End_Time=1463524604127, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:44,144 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524603903_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:44,144 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524603903_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:44 Completed Dag: dag_1463524603903_0001_1
2016-05-17 22:36:44,145 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524603903_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:44,145 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524603903_0001_1_post
2016-05-17 22:36:44,145 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:44,145 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:44,145 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524603903_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:36:44,145 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524603903_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:36:44,148 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524603903_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:44,148 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524603903_0001_00_000001 completed successfully
2016-05-17 22:36:44,149 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524603903_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:36:44,152 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...emblyplatform/0_ARGS_SWAP, applicationId=application_1463524603903_0001
2016-05-17 22:36:44,154 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:44,154 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:44,154 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:44,154 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:44,154 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:44,155 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:44,156 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:44,156 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:44,156 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:44,156 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:44,157 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:44,157 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:44,157 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:44,160 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1AD84105BE874DA28DAB39A4CFAD5224/EDBF602E60D84480B5C695655854420E/.tez/application_1463524603903_0001
2016-05-17 22:36:44,161 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/0_ARGS_SWAP/_temporary
2016-05-17 22:36:44,161 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.283
2016-05-17 22:36:44,231 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//0_ARGS_SWAP
2016-05-17 22:36:44,250 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:44,250 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:44,280 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 22:36:44,284 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 22:36:44,284 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:44,284 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:44,286 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:44,303 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:44,304 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_-1
2016-05-17 22:36:44,311 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:44,311 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:44,312 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:44,312 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:44,312 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_-1"]
2016-05-17 22:36:44,313 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:44,313 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:44,313 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:44,313 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...lyplatform/ALL_'field'_-1
2016-05-17 22:36:44,314 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:44,325 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:44,329 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:44,329 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:44,329 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:44,339 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/129B7F3911A14E689286B10C3B54AECA/E388C91BFEC740D4BBCDD5DCF537985C/.tez/application_1463524604329_0001 doesn't exist and is created
2016-05-17 22:36:44,356 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:44,356 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:44,357 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/129B7F3911A14E689286B10C3B54AECA/E388C91BFEC740D4BBCDD5DCF537985C/.tez/application_1463524604329_0001
2016-05-17 22:36:44,359 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524604329_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:44,368 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:44,369 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:44,369 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:44,370 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:44,370 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/129B7F3911A14E689286B10C3B54AECA/E388C91BFEC740D4BBCDD5DCF537985C/.tez/application_1463524604329_0001/localmode-log-dir
2016-05-17 22:36:44,370 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/129B7F3911A14E689286B10C3B54AECA/E388C91BFEC740D4BBCDD5DCF537985C/.tez/application_1463524604329_0001/localmode-log-dir/history.txt.appattempt_1463524604329_0001_000000, maxErrors=10
2016-05-17 22:36:44,370 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:44,370 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524604329_0001_000000, appSubmitTime=1463524604358, launchTime=1463524604359
2016-05-17 22:36:44,376 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:44,376 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:44,377 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:57480
2016-05-17 22:36:44,378 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:44,380 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:44,381 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:44,383 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524604329_0001_000000, startTime=1463524604383
2016-05-17 22:36:44,383 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:44,383 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:44,456 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:44,457 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:44,457 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_'field'_-1, applicationId=application_1463524604329_0001, dagName=[129B7F3911A14E689286B10C3B54AECA/E388C91BFEC740D4BBCDD5DCF537985C] each/(1/1) ...lyplatform/ALL_'field'_-1
2016-05-17 22:36:44,471 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [129B7F3911A14E689286B10C3B54AECA/E388C91BFEC740D4BBCDD5DCF537985C] each/(1/1) ...lyplatform/ALL_'field'_-1
2016-05-17 22:36:44,472 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524604329_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/129B7F3911A14E689286B10C3B54AECA/E388C91BFEC740D4BBCDD5DCF537985C/.tez/application_1463524604329_0001/localmode-log-dir/dag_1463524604329_0001_1.dot
2016-05-17 22:36:44,473 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/129B7F3911A14E689286B10C3B54AECA/E388C91BFEC740D4BBCDD5DCF537985C/.tez/application_1463524604329_0001/localmode-log-dir/dag_1463524604329_0001_1-tez-dag.pb.txt
2016-05-17 22:36:44,477 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524604329_0001_1
2016-05-17 22:36:44,477 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:44,477 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [129B7F3911A14E689286B10C3B54AECA/E388C91BFEC740D4BBCDD5DCF537985C] each/(1/1) ...lyplatform/ALL_'field'_-1
2016-05-17 22:36:44 Running Dag: dag_1463524604329_0001_1
2016-05-17 22:36:44,489 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604329_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524604329_0001_1, submitTime=1463524604471
2016-05-17 22:36:44,489 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:44,490 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B20AEF60C8D94DD69AC7A9E43D0AB594
2016-05-17 22:36:44,490 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:44,490 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604329_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524604329_0001_1, initTime=1463524604489
2016-05-17 22:36:44,490 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:44,491 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524604329_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:44,491 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604329_0001_1][Event:DAG_STARTED]: dagID=dag_1463524604329_0001_1, startTime=1463524604491
2016-05-17 22:36:44,491 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:44,491 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_'field'_-1, applicationId=application_1463524604329_0001, dagName=[129B7F3911A14E689286B10C3B54AECA/E388C91BFEC740D4BBCDD5DCF537985C] each/(1/1) ...lyplatform/ALL_'field'_-1
2016-05-17 22:36:44,491 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:44,491 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524604329_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:44,492 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B20AEF60C8D94DD69AC7A9E43D0AB594 : {810184962F474AF0B6F9D8A8954388AD={InputName=810184962F474AF0B6F9D8A8954388AD}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:44,492 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 810184962F474AF0B6F9D8A8954388AD, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:44,492 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594]
2016-05-17 22:36:44,492 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594]
2016-05-17 22:36:44,492 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594]
2016-05-17 22:36:44,492 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594]. Starting root input initializers: 1
2016-05-17 22:36:44,493 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:44,495 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 810184962F474AF0B6F9D8A8954388AD on vertex vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594]
2016-05-17 22:36:44,496 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:44,496 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 810184962F474AF0B6F9D8A8954388AD asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:44,497 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:44,497 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:44,498 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:44,498 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 810184962F474AF0B6F9D8A8954388AD on vertex vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594]
2016-05-17 22:36:44,498 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:44,499 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {810184962F474AF0B6F9D8A8954388AD=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:44,499 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594] parallelism set to 1
2016-05-17 22:36:44,499 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:44,499 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604329_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524604329_0001_1_00, eventCount=1
2016-05-17 22:36:44,499 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594]
2016-05-17 22:36:44,499 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=7FF39E0537E34250AD077BAD27E26357, vertexId=vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:44,499 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=7FF39E0537E34250AD077BAD27E26357, vertexId=vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594]
2016-05-17 22:36:44,501 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=7FF39E0537E34250AD077BAD27E26357, vertexName=B20AEF60C8D94DD69AC7A9E43D0AB594, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:44,501 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=7FF39E0537E34250AD077BAD27E26357, vertexId=vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594]
2016-05-17 22:36:44,502 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604329_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B20AEF60C8D94DD69AC7A9E43D0AB594, vertexId=vertex_1463524604329_0001_1_00, initRequestedTime=1463524604492, initedTime=1463524604502, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:44,502 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594] with distanceFromRoot: 0
2016-05-17 22:36:44,502 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:44,502 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:44,502 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B20AEF60C8D94DD69AC7A9E43D0AB594
2016-05-17 22:36:44,502 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594]
2016-05-17 22:36:44,503 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604329_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524604329_0001_1_00, startRequestedTime=1463524604502, startedTime=1463524604502
2016-05-17 22:36:44,503 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:44,503 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:44,504 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:44,504 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604329_0001_1][Event:TASK_STARTED]: vertexName=B20AEF60C8D94DD69AC7A9E43D0AB594, taskId=task_1463524604329_0001_1_00_000000, scheduledTime=1463524604504, launchTime=1463524604504
2016-05-17 22:36:44,504 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524604329_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:44,504 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594]
2016-05-17 22:36:44,504 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [129B7F3911A14E689286B10C3B54AECA/E388C91BFEC740D4BBCDD5DCF537985C] each/(1/1) ...lyplatform/ALL_'field'_-1, VertexName: B20AEF60C8D94DD69AC7A9E43D0AB594, VertexParallelism: 1, TaskAttemptID:attempt_1463524604329_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=810184962F474AF0B6F9D8A8954388AD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=7FF39E0537E34250AD077BAD27E26357, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:44,505 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524604329_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:44,505 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:44,505 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:44,506 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:44,506 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:44,506 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524604329_0001_00_000001
2016-05-17 22:36:44,506 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524604329_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:44,507 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524604329_0001_00_000001
2016-05-17 22:36:44,507 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:44,508 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524604329_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:44,508 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:44,508 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604329_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524604329_0001_00_000001, launchTime=1463524604508
2016-05-17 22:36:44,509 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524604329_0001_00_000001
2016-05-17 22:36:44,509 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524604329_0001_1_00_000000_0] to container: [container_1463524604329_0001_00_000001]
2016-05-17 22:36:44,510 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524604329_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:44,510 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524604329_0001_00_000001 given task: attempt_1463524604329_0001_1_00_000000_0
2016-05-17 22:36:44,510 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524604329_0001, with dag id: dag_1463524604329_0001_1
2016-05-17 22:36:44,510 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524604329_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524604329_0001_1_00_000000_0
2016-05-17 22:36:44,510 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524604329_0001_1_00_000000_0] started. Is using containerId: [container_1463524604329_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:44,511 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604329_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B20AEF60C8D94DD69AC7A9E43D0AB594, taskAttemptId=attempt_1463524604329_0001_1_00_000000_0, startTime=1463524604510, containerId=container_1463524604329_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524604329_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:44,511 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524604329_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:44,511 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:44,511 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524604329_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:44 Starting to run new task attempt: attempt_1463524604329_0001_1_00_000000_0
2016-05-17 22:36:44,512 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524604329_0001_1_00_000000_0
2016-05-17 22:36:44,512 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:44,512 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:44,512 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:44,512 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:44,513 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:44,513 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:44,513 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [129B7F3911A14E689286B10C3B54AECA/E388C91BFEC740D4BBCDD5DCF537985C] each/(1/1) ...lyplatform/ALL_'field'_-1, VertexName: B20AEF60C8D94DD69AC7A9E43D0AB594, VertexParallelism: 1, TaskAttemptID:attempt_1463524604329_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=810184962F474AF0B6F9D8A8954388AD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=7FF39E0537E34250AD077BAD27E26357, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:44,514 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:44,514 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524604329_0001_1_00_000000_0
2016-05-17 22:36:44,514 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:44,514 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:44,515 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:44,517 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=810184962F474AF0B6F9D8A8954388AD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:44,517 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:44,517 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 810184962F474AF0B6F9D8A8954388AD
2016-05-17 22:36:44,518 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 810184962F474AF0B6F9D8A8954388AD
2016-05-17 22:36:44,517 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524604329_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:44,519 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:44,520 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:44,520 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 810184962F474AF0B6F9D8A8954388AD
2016-05-17 22:36:44,520 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 810184962F474AF0B6F9D8A8954388AD
2016-05-17 22:36:44,520 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=7FF39E0537E34250AD077BAD27E26357, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:44,521 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:44,522 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 7FF39E0537E34250AD077BAD27E26357
2016-05-17 22:36:44,522 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:44,522 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 7FF39E0537E34250AD077BAD27E26357
2016-05-17 22:36:44,527 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B20AEF60C8D94DD69AC7A9E43D0AB594, ordinal: 0
2016-05-17 22:36:44,527 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B20AEF60C8D94DD69AC7A9E43D0AB594, mem on start (mb), free: 120, total: 350, max: 672
2016-05-17 22:36:44,527 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:44,527 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:44,528 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:44,532 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:44,532 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 7FF39E0537E34250AD077BAD27E26357
2016-05-17 22:36:44,532 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:44,532 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:44,533 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:44,533 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:44,533 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:44,533 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:44,533 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:44,533 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:44,533 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 810184962F474AF0B6F9D8A8954388AD, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:44,533 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 7FF39E0537E34250AD077BAD27E26357, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:44,533 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:44,533 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 810184962F474AF0B6F9D8A8954388AD being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:44,533 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 810184962F474AF0B6F9D8A8954388AD
2016-05-17 22:36:44,533 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:44,533 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 810184962F474AF0B6F9D8A8954388AD
2016-05-17 22:36:44,533 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:44,535 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:44,535 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:44,535 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524604329_0001_1_00_000000_0
2016-05-17 22:36:44,536 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:44,536 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2ad02ed4
2016-05-17 22:36:44,537 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2ad02ed4
2016-05-17 22:36:44,537 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:44,537 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:44,538 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:44,538 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 810184962F474AF0B6F9D8A8954388AD
2016-05-17 22:36:44,539 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_-1"], id: 7FF39E0537E34250AD077BAD27E26357
2016-05-17 22:36:44,539 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_-1"]
2016-05-17 22:36:44,539 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:44,539 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:44,539 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B20AEF60C8D94DD69AC7A9E43D0AB594, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:44,542 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:44,542 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:44,542 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524604329_0001_1_00_000000_0
2016-05-17 22:36:44,542 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524604329_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:44,543 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524604329_0001_1_00_000000_0
2016-05-17 22:36:44,543 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B20AEF60C8D94DD69AC7A9E43D0AB594, mem on close (mb), free: 119, total: 350, max: 672
2016-05-17 22:36:44 Completed running task attempt: attempt_1463524604329_0001_1_00_000000_0
2016-05-17 22:36:44,544 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524604329_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:44,545 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=367525888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524604539, Process_Duration=4, Process_End_Time=1463524604543, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:44,545 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:44,545 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:44,545 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604329_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B20AEF60C8D94DD69AC7A9E43D0AB594, taskAttemptId=attempt_1463524604329_0001_1_00_000000_0, startTime=1463524604510, finishTime=1463524604545, timeTaken=35, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=367525888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524604539, Process_Duration=4, Process_End_Time=1463524604543, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:44,545 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524604329_0001_00_000001
2016-05-17 22:36:44,545 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524604329_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:44,546 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:44,546 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:44,546 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524604329_0001_1_00_000000_0
2016-05-17 22:36:44,547 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604329_0001_1][Event:TASK_FINISHED]: vertexName=B20AEF60C8D94DD69AC7A9E43D0AB594, taskId=task_1463524604329_0001_1_00_000000, startTime=1463524604510, finishTime=1463524604546, timeTaken=36, status=SUCCEEDED, successfulAttemptID=attempt_1463524604329_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=367525888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524604539, Process_Duration=4, Process_End_Time=1463524604543, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:44,547 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524604329_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:44,547 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594] : 1
2016-05-17 22:36:44,547 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594]
2016-05-17 22:36:44,548 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:44,548 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604329_0001_1][Event:VERTEX_FINISHED]: vertexName=B20AEF60C8D94DD69AC7A9E43D0AB594, vertexId=vertex_1463524604329_0001_1_00, initRequestedTime=1463524604492, initedTime=1463524604502, startRequestedTime=1463524604502, startedTime=1463524604502, finishTime=1463524604547, timeTaken=45, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=367525888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524604539, Process_Duration=4, Process_End_Time=1463524604543, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524604510, firstTasksToStart=[ task_1463524604329_0001_1_00_000000 ], lastTaskFinishTime=1463524604545, lastTasksToFinish=[ task_1463524604329_0001_1_00_000000 ], minTaskDuration=35, maxTaskDuration=35, avgTaskDuration=35.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524604329_0001_1_00_000000 ], longestDurationTasks=[ task_1463524604329_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:44,548 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:44,548 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:44,548 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524604329_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:44,549 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604329_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524604329_0001_00_000001, stoppedTime=1463524604549, exitStatus=0
2016-05-17 22:36:44,549 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524604329_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:44,549 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:44,550 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524604329_0001_1
2016-05-17 22:36:44,550 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604329_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524604329_0001_1
2016-05-17 22:36:44,550 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 7FF39E0537E34250AD077BAD27E26357 for vertex: vertex_1463524604329_0001_1_00 [B20AEF60C8D94DD69AC7A9E43D0AB594]
2016-05-17 22:36:44,550 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:44,556 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524604329_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:44,556 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524604329_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:44,556 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524604329_0001_00_000001, Exiting
2016-05-17 22:36:44,556 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524604329_0001_00_000001
2016-05-17 22:36:44,556 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524604329_0001_00_000001
2016-05-17 22:36:44,557 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524604329_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:44,557 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524604329_0001_00_000001 completed successfully
2016-05-17 22:36:44,558 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524604329_0001_1, queueSize=0
2016-05-17 22:36:44,558 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:44,558 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604329_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524604329_0001_1, startTime=1463524604491, finishTime=1463524604556, timeTaken=65, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=367525888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524604539, Process_Duration=4, Process_End_Time=1463524604543, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:44,558 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524604329_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:44,558 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524604329_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:44,558 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524604329_0001_00_000001
2016-05-17 22:36:44,558 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524604329_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:44 Completed Dag: dag_1463524604329_0001_1
2016-05-17 22:36:44,560 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524604329_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:44,560 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524604329_0001_1_post
2016-05-17 22:36:44,560 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:44,560 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:44,566 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/ALL_'field'_-1, applicationId=application_1463524604329_0001
2016-05-17 22:36:44,567 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:44,567 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:44,568 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:44,568 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:44,568 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:44,568 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:44,568 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:44,568 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:44,568 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:44,568 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:44,569 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:44,569 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:44,570 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:44,573 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/129B7F3911A14E689286B10C3B54AECA/E388C91BFEC740D4BBCDD5DCF537985C/.tez/application_1463524604329_0001
2016-05-17 22:36:44,573 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_-1/_temporary
2016-05-17 22:36:44,574 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.261
2016-05-17 22:36:44,637 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_'field'_-1
2016-05-17 22:36:44,669 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:44,670 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:44,697 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-05-17 22:36:44,701 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 22:36:44,702 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:44,702 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:44,702 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:44,714 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:44,715 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_'field'
2016-05-17 22:36:44,724 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:44,724 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:44,724 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_'field'"]
2016-05-17 22:36:44,724 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:44,724 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:44,724 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:44,725 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...tform/ALL_'field'_'field'
2016-05-17 22:36:44,725 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:44,728 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:44,728 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:44,736 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:44,741 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:44,741 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:44,741 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:44,750 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8A8DC594BCDB470BA7669974EA52E6B0/B824D525C94C49AF897CE37B61545AFF/.tez/application_1463524604741_0001 doesn't exist and is created
2016-05-17 22:36:44,765 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:44,765 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:44,766 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8A8DC594BCDB470BA7669974EA52E6B0/B824D525C94C49AF897CE37B61545AFF/.tez/application_1463524604741_0001
2016-05-17 22:36:44,768 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524604741_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:44,777 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:44,778 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:44,778 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:44,779 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:44,779 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8A8DC594BCDB470BA7669974EA52E6B0/B824D525C94C49AF897CE37B61545AFF/.tez/application_1463524604741_0001/localmode-log-dir
2016-05-17 22:36:44,779 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8A8DC594BCDB470BA7669974EA52E6B0/B824D525C94C49AF897CE37B61545AFF/.tez/application_1463524604741_0001/localmode-log-dir/history.txt.appattempt_1463524604741_0001_000000, maxErrors=10
2016-05-17 22:36:44,779 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:44,779 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524604741_0001_000000, appSubmitTime=1463524604767, launchTime=1463524604768
2016-05-17 22:36:44,782 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:44,784 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:44,785 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:44,786 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:44,787 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:54277
2016-05-17 22:36:44,789 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:44,790 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524604741_0001_000000, startTime=1463524604790
2016-05-17 22:36:44,790 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:44,790 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:44,866 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:44,866 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:44,866 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/ALL_'field'_'field', applicationId=application_1463524604741_0001, dagName=[8A8DC594BCDB470BA7669974EA52E6B0/B824D525C94C49AF897CE37B61545AFF] each/(1/1) ...tform/ALL_'field'_'field'
2016-05-17 22:36:44,880 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [8A8DC594BCDB470BA7669974EA52E6B0/B824D525C94C49AF897CE37B61545AFF] each/(1/1) ...tform/ALL_'field'_'field'
2016-05-17 22:36:44,882 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524604741_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8A8DC594BCDB470BA7669974EA52E6B0/B824D525C94C49AF897CE37B61545AFF/.tez/application_1463524604741_0001/localmode-log-dir/dag_1463524604741_0001_1.dot
2016-05-17 22:36:44,882 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8A8DC594BCDB470BA7669974EA52E6B0/B824D525C94C49AF897CE37B61545AFF/.tez/application_1463524604741_0001/localmode-log-dir/dag_1463524604741_0001_1-tez-dag.pb.txt
2016-05-17 22:36:44,886 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524604741_0001_1
2016-05-17 22:36:44,886 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:44,886 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [8A8DC594BCDB470BA7669974EA52E6B0/B824D525C94C49AF897CE37B61545AFF] each/(1/1) ...tform/ALL_'field'_'field'
2016-05-17 22:36:44 Running Dag: dag_1463524604741_0001_1
2016-05-17 22:36:44,896 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604741_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524604741_0001_1, submitTime=1463524604881
2016-05-17 22:36:44,896 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:44,897 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D1799F01474744759A201E5D4B38DC23
2016-05-17 22:36:44,897 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:44,897 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604741_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524604741_0001_1, initTime=1463524604896
2016-05-17 22:36:44,897 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524604741_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:44,898 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604741_0001_1][Event:DAG_STARTED]: dagID=dag_1463524604741_0001_1, startTime=1463524604897
2016-05-17 22:36:44,898 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:44,898 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524604741_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:44,898 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D1799F01474744759A201E5D4B38DC23 : {0623E55C4EA24D5D8236040CFF087B3F={InputName=0623E55C4EA24D5D8236040CFF087B3F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:44,898 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 0623E55C4EA24D5D8236040CFF087B3F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:44,898 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23]
2016-05-17 22:36:44,898 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23]
2016-05-17 22:36:44,898 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23]
2016-05-17 22:36:44,897 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:44,899 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:44,899 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23]. Starting root input initializers: 1
2016-05-17 22:36:44,899 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:44,899 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 0623E55C4EA24D5D8236040CFF087B3F on vertex vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23]
2016-05-17 22:36:44,900 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/ALL_'field'_'field', applicationId=application_1463524604741_0001, dagName=[8A8DC594BCDB470BA7669974EA52E6B0/B824D525C94C49AF897CE37B61545AFF] each/(1/1) ...tform/ALL_'field'_'field'
2016-05-17 22:36:44,902 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:44,902 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 0623E55C4EA24D5D8236040CFF087B3F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:44,902 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:44,903 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:44,904 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:44,904 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 0623E55C4EA24D5D8236040CFF087B3F on vertex vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23]
2016-05-17 22:36:44,905 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:44,905 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {0623E55C4EA24D5D8236040CFF087B3F=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:44,905 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23] parallelism set to 1
2016-05-17 22:36:44,905 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:44,905 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604741_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524604741_0001_1_00, eventCount=1
2016-05-17 22:36:44,905 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23]
2016-05-17 22:36:44,905 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=78CA82702E0B40F7A8A5C51E38766010, vertexId=vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:44,906 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=78CA82702E0B40F7A8A5C51E38766010, vertexId=vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23]
2016-05-17 22:36:44,907 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=78CA82702E0B40F7A8A5C51E38766010, vertexName=D1799F01474744759A201E5D4B38DC23, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:44,908 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=78CA82702E0B40F7A8A5C51E38766010, vertexId=vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23]
2016-05-17 22:36:44,909 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604741_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D1799F01474744759A201E5D4B38DC23, vertexId=vertex_1463524604741_0001_1_00, initRequestedTime=1463524604898, initedTime=1463524604909, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:44,909 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23] with distanceFromRoot: 0
2016-05-17 22:36:44,909 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:44,909 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:44,909 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D1799F01474744759A201E5D4B38DC23
2016-05-17 22:36:44,909 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23]
2016-05-17 22:36:44,910 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:44,910 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604741_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524604741_0001_1_00, startRequestedTime=1463524604909, startedTime=1463524604909
2016-05-17 22:36:44,910 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:44,910 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:44,911 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604741_0001_1][Event:TASK_STARTED]: vertexName=D1799F01474744759A201E5D4B38DC23, taskId=task_1463524604741_0001_1_00_000000, scheduledTime=1463524604910, launchTime=1463524604910
2016-05-17 22:36:44,911 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524604741_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:44,911 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23]
2016-05-17 22:36:44,911 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8A8DC594BCDB470BA7669974EA52E6B0/B824D525C94C49AF897CE37B61545AFF] each/(1/1) ...tform/ALL_'field'_'field', VertexName: D1799F01474744759A201E5D4B38DC23, VertexParallelism: 1, TaskAttemptID:attempt_1463524604741_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0623E55C4EA24D5D8236040CFF087B3F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=78CA82702E0B40F7A8A5C51E38766010, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:44,912 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524604741_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:44,912 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:44,912 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:44,913 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:44,913 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:44,913 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524604741_0001_00_000001
2016-05-17 22:36:44,913 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524604741_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:44,915 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524604741_0001_00_000001
2016-05-17 22:36:44,915 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:44,916 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524604741_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:44,916 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604741_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524604741_0001_00_000001, launchTime=1463524604916
2016-05-17 22:36:44,916 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:44,916 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524604741_0001_00_000001
2016-05-17 22:36:44,917 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524604741_0001_1_00_000000_0] to container: [container_1463524604741_0001_00_000001]
2016-05-17 22:36:44,917 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524604741_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:44,918 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524604741_0001_1_00_000000_0] started. Is using containerId: [container_1463524604741_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:44,918 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604741_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D1799F01474744759A201E5D4B38DC23, taskAttemptId=attempt_1463524604741_0001_1_00_000000_0, startTime=1463524604917, containerId=container_1463524604741_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524604741_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:44,918 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524604741_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:44,918 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524604741_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:44,918 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:44,919 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524604741_0001_00_000001 given task: attempt_1463524604741_0001_1_00_000000_0
2016-05-17 22:36:44,919 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524604741_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524604741_0001_1_00_000000_0
2016-05-17 22:36:44 Starting to run new task attempt: attempt_1463524604741_0001_1_00_000000_0
2016-05-17 22:36:44,919 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524604741_0001_1_00_000000_0
2016-05-17 22:36:44,919 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:44,920 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:44,920 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:44,920 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524604741_0001, with dag id: dag_1463524604741_0001_1
2016-05-17 22:36:44,920 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:44,920 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:44,920 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:44,920 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8A8DC594BCDB470BA7669974EA52E6B0/B824D525C94C49AF897CE37B61545AFF] each/(1/1) ...tform/ALL_'field'_'field', VertexName: D1799F01474744759A201E5D4B38DC23, VertexParallelism: 1, TaskAttemptID:attempt_1463524604741_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0623E55C4EA24D5D8236040CFF087B3F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=78CA82702E0B40F7A8A5C51E38766010, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:44,921 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:44,921 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524604741_0001_1_00_000000_0
2016-05-17 22:36:44,921 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:44,921 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:44,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=0623E55C4EA24D5D8236040CFF087B3F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:44,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:44,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:44,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 0623E55C4EA24D5D8236040CFF087B3F
2016-05-17 22:36:44,923 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 0623E55C4EA24D5D8236040CFF087B3F
2016-05-17 22:36:44,925 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:44,925 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 0623E55C4EA24D5D8236040CFF087B3F
2016-05-17 22:36:44,925 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 0623E55C4EA24D5D8236040CFF087B3F
2016-05-17 22:36:44,922 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524604741_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:44,926 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:44,926 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=78CA82702E0B40F7A8A5C51E38766010, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:44,926 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:44,926 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 78CA82702E0B40F7A8A5C51E38766010
2016-05-17 22:36:44,927 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:44,927 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 78CA82702E0B40F7A8A5C51E38766010
2016-05-17 22:36:44,932 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D1799F01474744759A201E5D4B38DC23, ordinal: 0
2016-05-17 22:36:44,932 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D1799F01474744759A201E5D4B38DC23, mem on start (mb), free: 220, total: 341, max: 672
2016-05-17 22:36:44,933 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:44,933 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:44,933 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:44,937 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:44,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 78CA82702E0B40F7A8A5C51E38766010
2016-05-17 22:36:44,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:44,938 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:44,938 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:44,938 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:44,938 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:44,938 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:44,939 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:44,939 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:44,939 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 0623E55C4EA24D5D8236040CFF087B3F, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:44,939 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 78CA82702E0B40F7A8A5C51E38766010, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:44,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:44,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 0623E55C4EA24D5D8236040CFF087B3F being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:44,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:44,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:44,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 0623E55C4EA24D5D8236040CFF087B3F
2016-05-17 22:36:44,939 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:44,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 0623E55C4EA24D5D8236040CFF087B3F
2016-05-17 22:36:44,940 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:44,940 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524604741_0001_1_00_000000_0
2016-05-17 22:36:44,941 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:44,941 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@59ef40eb
2016-05-17 22:36:44,942 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@59ef40eb
2016-05-17 22:36:44,942 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:44,942 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:44,943 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:44,943 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 0623E55C4EA24D5D8236040CFF087B3F
2016-05-17 22:36:44,944 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_'field'"], id: 78CA82702E0B40F7A8A5C51E38766010
2016-05-17 22:36:44,944 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_'field'"]
2016-05-17 22:36:44,944 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:44,944 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:44,944 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D1799F01474744759A201E5D4B38DC23, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:44,948 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:44,948 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:44,948 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524604741_0001_1_00_000000_0
2016-05-17 22:36:44,948 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524604741_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:44,949 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524604741_0001_1_00_000000_0
2016-05-17 22:36:44,949 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D1799F01474744759A201E5D4B38DC23, mem on close (mb), free: 218, total: 341, max: 672
2016-05-17 22:36:44 Completed running task attempt: attempt_1463524604741_0001_1_00_000000_0
2016-05-17 22:36:44,950 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524604741_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:44,951 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604741_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D1799F01474744759A201E5D4B38DC23, taskAttemptId=attempt_1463524604741_0001_1_00_000000_0, startTime=1463524604917, finishTime=1463524604951, timeTaken=34, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=357564416, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524604944, Process_Duration=5, Process_End_Time=1463524604949, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:44,951 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:44,951 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524604741_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:44,951 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524604741_0001_1_00_000000_0
2016-05-17 22:36:44,952 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604741_0001_1][Event:TASK_FINISHED]: vertexName=D1799F01474744759A201E5D4B38DC23, taskId=task_1463524604741_0001_1_00_000000, startTime=1463524604917, finishTime=1463524604952, timeTaken=35, status=SUCCEEDED, successfulAttemptID=attempt_1463524604741_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=357564416, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524604944, Process_Duration=5, Process_End_Time=1463524604949, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:44,952 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:44,952 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524604741_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:44,952 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:44,953 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23] : 1
2016-05-17 22:36:44,953 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23]
2016-05-17 22:36:44,954 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=357564416, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524604944, Process_Duration=5, Process_End_Time=1463524604949, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:44,954 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:44,954 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604741_0001_1][Event:VERTEX_FINISHED]: vertexName=D1799F01474744759A201E5D4B38DC23, vertexId=vertex_1463524604741_0001_1_00, initRequestedTime=1463524604898, initedTime=1463524604909, startRequestedTime=1463524604909, startedTime=1463524604909, finishTime=1463524604953, timeTaken=44, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=357564416, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524604944, Process_Duration=5, Process_End_Time=1463524604949, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524604917, firstTasksToStart=[ task_1463524604741_0001_1_00_000000 ], lastTaskFinishTime=1463524604951, lastTasksToFinish=[ task_1463524604741_0001_1_00_000000 ], minTaskDuration=34, maxTaskDuration=34, avgTaskDuration=34.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524604741_0001_1_00_000000 ], longestDurationTasks=[ task_1463524604741_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:44,954 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:44,955 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524604741_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:44,955 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604741_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524604741_0001_00_000001, stoppedTime=1463524604955, exitStatus=0
2016-05-17 22:36:44,955 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524604741_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:44,955 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:44,955 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524604741_0001_1
2016-05-17 22:36:44,956 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604741_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524604741_0001_1
2016-05-17 22:36:44,956 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 78CA82702E0B40F7A8A5C51E38766010 for vertex: vertex_1463524604741_0001_1_00 [D1799F01474744759A201E5D4B38DC23]
2016-05-17 22:36:44,955 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:44,955 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:44,957 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:44,957 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524604741_0001_00_000001
2016-05-17 22:36:44,957 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524604741_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:44,957 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524604741_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:44,958 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524604741_0001_00_000001, Exiting
2016-05-17 22:36:44,958 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524604741_0001_00_000001
2016-05-17 22:36:44,958 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524604741_0001_00_000001
2016-05-17 22:36:44,959 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524604741_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:44,959 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524604741_0001_00_000001 completed successfully
2016-05-17 22:36:44,964 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524604741_0001_1, queueSize=0
2016-05-17 22:36:44,964 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:44,965 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524604741_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524604741_0001_1, startTime=1463524604897, finishTime=1463524604963, timeTaken=66, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=357564416, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524604944, Process_Duration=5, Process_End_Time=1463524604949, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:44,965 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524604741_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:44,965 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524604741_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:44,965 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524604741_0001_00_000001
2016-05-17 22:36:44,965 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524604741_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:44 Completed Dag: dag_1463524604741_0001_1
2016-05-17 22:36:44,966 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524604741_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:44,966 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524604741_0001_1_post
2016-05-17 22:36:44,966 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:44,966 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:44,971 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/ALL_'field'_'field', applicationId=application_1463524604741_0001
2016-05-17 22:36:44,974 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:44,974 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:44,974 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:44,974 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:44,974 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:44,975 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:44,975 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:44,975 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:44,975 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:44,975 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:44,975 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:44,976 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:44,976 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:44,980 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8A8DC594BCDB470BA7669974EA52E6B0/B824D525C94C49AF897CE37B61545AFF/.tez/application_1463524604741_0001
2016-05-17 22:36:44,980 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_'field'/_temporary
2016-05-17 22:36:44,981 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.256
2016-05-17 22:36:45,046 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_'field'_'field'
2016-05-17 22:36:45,067 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:45,068 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:45,098 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-05-17 22:36:45,102 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 22:36:45,102 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:45,103 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:45,103 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:45,117 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:45,118 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_RESULTS
2016-05-17 22:36:45,125 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:45,125 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:45,125 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_RESULTS"]
2016-05-17 22:36:45,126 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:45,126 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:45,126 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:45,127 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...tform/ALL_'field'_RESULTS
2016-05-17 22:36:45,127 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:45,139 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:45,143 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:45,144 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:45,144 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:45,153 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E90D0A635D414BDE99F41638F7E929EC/AEDD6B11F775470A9FA010C1BCE2A449/.tez/application_1463524605143_0001 doesn't exist and is created
2016-05-17 22:36:45,155 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:45,155 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:45,169 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:45,169 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:45,169 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E90D0A635D414BDE99F41638F7E929EC/AEDD6B11F775470A9FA010C1BCE2A449/.tez/application_1463524605143_0001
2016-05-17 22:36:45,171 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524605143_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:45,181 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:45,182 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:45,182 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:45,183 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:45,183 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E90D0A635D414BDE99F41638F7E929EC/AEDD6B11F775470A9FA010C1BCE2A449/.tez/application_1463524605143_0001/localmode-log-dir
2016-05-17 22:36:45,183 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E90D0A635D414BDE99F41638F7E929EC/AEDD6B11F775470A9FA010C1BCE2A449/.tez/application_1463524605143_0001/localmode-log-dir/history.txt.appattempt_1463524605143_0001_000000, maxErrors=10
2016-05-17 22:36:45,183 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:45,183 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524605143_0001_000000, appSubmitTime=1463524605171, launchTime=1463524605171
2016-05-17 22:36:45,187 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:45,187 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:41815
2016-05-17 22:36:45,188 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:45,190 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:45,190 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:45,191 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:45,192 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524605143_0001_000000, startTime=1463524605192
2016-05-17 22:36:45,192 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:45,192 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:45,269 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:45,269 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:45,269 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/ALL_'field'_RESULTS, applicationId=application_1463524605143_0001, dagName=[E90D0A635D414BDE99F41638F7E929EC/AEDD6B11F775470A9FA010C1BCE2A449] each/(1/1) ...tform/ALL_'field'_RESULTS
2016-05-17 22:36:45,283 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [E90D0A635D414BDE99F41638F7E929EC/AEDD6B11F775470A9FA010C1BCE2A449] each/(1/1) ...tform/ALL_'field'_RESULTS
2016-05-17 22:36:45,284 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524605143_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E90D0A635D414BDE99F41638F7E929EC/AEDD6B11F775470A9FA010C1BCE2A449/.tez/application_1463524605143_0001/localmode-log-dir/dag_1463524605143_0001_1.dot
2016-05-17 22:36:45,285 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E90D0A635D414BDE99F41638F7E929EC/AEDD6B11F775470A9FA010C1BCE2A449/.tez/application_1463524605143_0001/localmode-log-dir/dag_1463524605143_0001_1-tez-dag.pb.txt
2016-05-17 22:36:45,288 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524605143_0001_1
2016-05-17 22:36:45,289 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:45,289 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [E90D0A635D414BDE99F41638F7E929EC/AEDD6B11F775470A9FA010C1BCE2A449] each/(1/1) ...tform/ALL_'field'_RESULTS
2016-05-17 22:36:45 Running Dag: dag_1463524605143_0001_1
2016-05-17 22:36:45,300 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605143_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524605143_0001_1, submitTime=1463524605283
2016-05-17 22:36:45,300 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:45,300 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C7D097C778FC4F92853E170A574AD702
2016-05-17 22:36:45,300 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:45,301 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605143_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524605143_0001_1, initTime=1463524605300
2016-05-17 22:36:45,301 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:45,301 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524605143_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:45,301 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605143_0001_1][Event:DAG_STARTED]: dagID=dag_1463524605143_0001_1, startTime=1463524605301
2016-05-17 22:36:45,302 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:45,302 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524605143_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:45,302 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C7D097C778FC4F92853E170A574AD702 : {710E499C792946B28E65289B31AE2AB9={InputName=710E499C792946B28E65289B31AE2AB9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:45,302 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 710E499C792946B28E65289B31AE2AB9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:45,302 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702]
2016-05-17 22:36:45,302 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:45,302 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702]
2016-05-17 22:36:45,302 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702]
2016-05-17 22:36:45,302 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/ALL_'field'_RESULTS, applicationId=application_1463524605143_0001, dagName=[E90D0A635D414BDE99F41638F7E929EC/AEDD6B11F775470A9FA010C1BCE2A449] each/(1/1) ...tform/ALL_'field'_RESULTS
2016-05-17 22:36:45,303 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702]. Starting root input initializers: 1
2016-05-17 22:36:45,303 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:45,303 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 710E499C792946B28E65289B31AE2AB9 on vertex vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702]
2016-05-17 22:36:45,305 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:45,305 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 710E499C792946B28E65289B31AE2AB9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:45,306 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:45,306 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:45,307 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:45,307 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 710E499C792946B28E65289B31AE2AB9 on vertex vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702]
2016-05-17 22:36:45,308 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:45,308 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {710E499C792946B28E65289B31AE2AB9=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:45,308 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702] parallelism set to 1
2016-05-17 22:36:45,308 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:45,308 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605143_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524605143_0001_1_00, eventCount=1
2016-05-17 22:36:45,308 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702]
2016-05-17 22:36:45,308 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=173DA49DC0744702A1BFEFB61A11C4C7, vertexId=vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:45,309 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=173DA49DC0744702A1BFEFB61A11C4C7, vertexId=vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702]
2016-05-17 22:36:45,310 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=173DA49DC0744702A1BFEFB61A11C4C7, vertexName=C7D097C778FC4F92853E170A574AD702, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:45,311 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=173DA49DC0744702A1BFEFB61A11C4C7, vertexId=vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702]
2016-05-17 22:36:45,312 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:45,312 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605143_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C7D097C778FC4F92853E170A574AD702, vertexId=vertex_1463524605143_0001_1_00, initRequestedTime=1463524605302, initedTime=1463524605311, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:45,312 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702] with distanceFromRoot: 0
2016-05-17 22:36:45,312 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:45,312 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C7D097C778FC4F92853E170A574AD702
2016-05-17 22:36:45,312 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702]
2016-05-17 22:36:45,313 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:45,313 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605143_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524605143_0001_1_00, startRequestedTime=1463524605312, startedTime=1463524605312
2016-05-17 22:36:45,313 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:45,313 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:45,314 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605143_0001_1][Event:TASK_STARTED]: vertexName=C7D097C778FC4F92853E170A574AD702, taskId=task_1463524605143_0001_1_00_000000, scheduledTime=1463524605313, launchTime=1463524605313
2016-05-17 22:36:45,314 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524605143_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:45,314 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702]
2016-05-17 22:36:45,314 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E90D0A635D414BDE99F41638F7E929EC/AEDD6B11F775470A9FA010C1BCE2A449] each/(1/1) ...tform/ALL_'field'_RESULTS, VertexName: C7D097C778FC4F92853E170A574AD702, VertexParallelism: 1, TaskAttemptID:attempt_1463524605143_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=710E499C792946B28E65289B31AE2AB9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=173DA49DC0744702A1BFEFB61A11C4C7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:45,315 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524605143_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:45,315 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:45,315 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:45,316 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:45,316 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:45,316 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524605143_0001_00_000001
2016-05-17 22:36:45,316 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524605143_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:45,317 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524605143_0001_00_000001
2016-05-17 22:36:45,317 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:45,318 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524605143_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:45,318 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:45,319 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605143_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524605143_0001_00_000001, launchTime=1463524605318
2016-05-17 22:36:45,319 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524605143_0001_00_000001
2016-05-17 22:36:45,320 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524605143_0001_1_00_000000_0] to container: [container_1463524605143_0001_00_000001]
2016-05-17 22:36:45,320 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524605143_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:45,320 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524605143_0001_1_00_000000_0] started. Is using containerId: [container_1463524605143_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:45,320 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605143_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C7D097C778FC4F92853E170A574AD702, taskAttemptId=attempt_1463524605143_0001_1_00_000000_0, startTime=1463524605320, containerId=container_1463524605143_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524605143_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:45,320 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524605143_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:45,321 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524605143_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:45,321 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:45,321 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524605143_0001_00_000001 given task: attempt_1463524605143_0001_1_00_000000_0
2016-05-17 22:36:45,321 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524605143_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524605143_0001_1_00_000000_0
2016-05-17 22:36:45 Starting to run new task attempt: attempt_1463524605143_0001_1_00_000000_0
2016-05-17 22:36:45,322 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524605143_0001_1_00_000000_0
2016-05-17 22:36:45,322 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:45,322 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:45,322 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:45,322 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:45,322 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:45,322 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:45,322 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E90D0A635D414BDE99F41638F7E929EC/AEDD6B11F775470A9FA010C1BCE2A449] each/(1/1) ...tform/ALL_'field'_RESULTS, VertexName: C7D097C778FC4F92853E170A574AD702, VertexParallelism: 1, TaskAttemptID:attempt_1463524605143_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=710E499C792946B28E65289B31AE2AB9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=173DA49DC0744702A1BFEFB61A11C4C7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:45,323 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:45,323 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524605143_0001, with dag id: dag_1463524605143_0001_1
2016-05-17 22:36:45,325 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524605143_0001_1_00_000000_0
2016-05-17 22:36:45,325 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:45,325 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:45,325 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524605143_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:45,326 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:45,326 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=710E499C792946B28E65289B31AE2AB9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:45,326 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:45,326 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 710E499C792946B28E65289B31AE2AB9
2016-05-17 22:36:45,326 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 710E499C792946B28E65289B31AE2AB9
2016-05-17 22:36:45,328 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:45,329 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:45,329 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 710E499C792946B28E65289B31AE2AB9
2016-05-17 22:36:45,330 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 710E499C792946B28E65289B31AE2AB9
2016-05-17 22:36:45,328 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=173DA49DC0744702A1BFEFB61A11C4C7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:45,330 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:45,330 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 173DA49DC0744702A1BFEFB61A11C4C7
2016-05-17 22:36:45,330 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:45,331 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 173DA49DC0744702A1BFEFB61A11C4C7
2016-05-17 22:36:45,335 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C7D097C778FC4F92853E170A574AD702, ordinal: 0
2016-05-17 22:36:45,335 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C7D097C778FC4F92853E170A574AD702, mem on start (mb), free: 144, total: 341, max: 672
2016-05-17 22:36:45,335 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:45,336 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:45,336 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:45,340 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:45,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 173DA49DC0744702A1BFEFB61A11C4C7
2016-05-17 22:36:45,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:45,340 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:45,340 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:45,340 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:45,340 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:45,340 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:45,341 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:45,341 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:45,341 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 710E499C792946B28E65289B31AE2AB9, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:45,341 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 173DA49DC0744702A1BFEFB61A11C4C7, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:45,341 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:45,341 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 710E499C792946B28E65289B31AE2AB9 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:45,341 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:45,341 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:45,341 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 710E499C792946B28E65289B31AE2AB9
2016-05-17 22:36:45,341 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:45,341 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 710E499C792946B28E65289B31AE2AB9
2016-05-17 22:36:45,342 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:45,342 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524605143_0001_1_00_000000_0
2016-05-17 22:36:45,344 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:45,345 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1a0e6a46
2016-05-17 22:36:45,345 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:45,346 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 710E499C792946B28E65289B31AE2AB9
2016-05-17 22:36:45,346 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_RESULTS"], id: 173DA49DC0744702A1BFEFB61A11C4C7
2016-05-17 22:36:45,346 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_RESULTS"]
2016-05-17 22:36:45,347 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:45,347 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:45,346 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1a0e6a46
2016-05-17 22:36:45,347 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:45,347 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:45,347 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C7D097C778FC4F92853E170A574AD702, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:45,349 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:45,349 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:45,350 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524605143_0001_1_00_000000_0
2016-05-17 22:36:45,350 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524605143_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:45,350 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524605143_0001_1_00_000000_0
2016-05-17 22:36:45,350 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C7D097C778FC4F92853E170A574AD702, mem on close (mb), free: 142, total: 341, max: 672
2016-05-17 22:36:45 Completed running task attempt: attempt_1463524605143_0001_1_00_000000_0
2016-05-17 22:36:45,351 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524605143_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:45,351 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:45,351 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605143_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C7D097C778FC4F92853E170A574AD702, taskAttemptId=attempt_1463524605143_0001_1_00_000000_0, startTime=1463524605320, finishTime=1463524605351, timeTaken=31, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=357564416, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524605347, Process_Duration=3, Process_End_Time=1463524605350, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:45,351 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524605143_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:45,352 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524605143_0001_1_00_000000_0
2016-05-17 22:36:45,352 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:45,352 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:45,352 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605143_0001_1][Event:TASK_FINISHED]: vertexName=C7D097C778FC4F92853E170A574AD702, taskId=task_1463524605143_0001_1_00_000000, startTime=1463524605320, finishTime=1463524605352, timeTaken=32, status=SUCCEEDED, successfulAttemptID=attempt_1463524605143_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=357564416, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524605347, Process_Duration=3, Process_End_Time=1463524605350, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:45,352 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524605143_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:45,352 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702] : 1
2016-05-17 22:36:45,352 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702]
2016-05-17 22:36:45,353 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:45,353 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605143_0001_1][Event:VERTEX_FINISHED]: vertexName=C7D097C778FC4F92853E170A574AD702, vertexId=vertex_1463524605143_0001_1_00, initRequestedTime=1463524605302, initedTime=1463524605311, startRequestedTime=1463524605312, startedTime=1463524605312, finishTime=1463524605352, timeTaken=40, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=357564416, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524605347, Process_Duration=3, Process_End_Time=1463524605350, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524605320, firstTasksToStart=[ task_1463524605143_0001_1_00_000000 ], lastTaskFinishTime=1463524605351, lastTasksToFinish=[ task_1463524605143_0001_1_00_000000 ], minTaskDuration=31, maxTaskDuration=31, avgTaskDuration=31.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524605143_0001_1_00_000000 ], longestDurationTasks=[ task_1463524605143_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:45,353 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:45,353 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524605143_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:45,353 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605143_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524605143_0001_00_000001, stoppedTime=1463524605353, exitStatus=0
2016-05-17 22:36:45,353 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:45,354 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524605143_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:45,354 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:45,354 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524605143_0001_1
2016-05-17 22:36:45,354 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605143_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524605143_0001_1
2016-05-17 22:36:45,354 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 173DA49DC0744702A1BFEFB61A11C4C7 for vertex: vertex_1463524605143_0001_1_00 [C7D097C778FC4F92853E170A574AD702]
2016-05-17 22:36:45,355 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=357564416, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524605347, Process_Duration=3, Process_End_Time=1463524605350, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:45,356 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:45,356 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:45,356 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524605143_0001_00_000001
2016-05-17 22:36:45,356 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524605143_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:45,357 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524605143_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:45,357 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524605143_0001_00_000001, Exiting
2016-05-17 22:36:45,357 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524605143_0001_00_000001
2016-05-17 22:36:45,357 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524605143_0001_00_000001
2016-05-17 22:36:45,359 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524605143_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:45,359 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524605143_0001_00_000001 completed successfully
2016-05-17 22:36:45,365 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524605143_0001_1, queueSize=0
2016-05-17 22:36:45,365 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:45,365 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605143_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524605143_0001_1, startTime=1463524605301, finishTime=1463524605364, timeTaken=63, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=357564416, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524605347, Process_Duration=3, Process_End_Time=1463524605350, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:45,365 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524605143_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:45,365 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524605143_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:45,365 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524605143_0001_00_000001
2016-05-17 22:36:45,366 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524605143_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:45 Completed Dag: dag_1463524605143_0001_1
2016-05-17 22:36:45,366 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524605143_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:45,366 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524605143_0001_1_post
2016-05-17 22:36:45,367 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:45,367 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:45,371 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/ALL_'field'_RESULTS, applicationId=application_1463524605143_0001
2016-05-17 22:36:45,373 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:45,373 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:45,373 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:45,373 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:45,373 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:45,373 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:45,374 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:45,374 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:45,374 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:45,374 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:45,374 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:45,375 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:45,375 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:45,380 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E90D0A635D414BDE99F41638F7E929EC/AEDD6B11F775470A9FA010C1BCE2A449/.tez/application_1463524605143_0001
2016-05-17 22:36:45,381 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_RESULTS/_temporary
2016-05-17 22:36:45,382 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.255
2016-05-17 22:36:45,449 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_'field'_RESULTS
2016-05-17 22:36:45,470 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:45,470 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:45,502 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-05-17 22:36:45,504 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 22:36:45,504 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:45,504 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:45,504 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:45,519 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:45,520 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_ALL
2016-05-17 22:36:45,528 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:45,528 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:45,528 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_ALL"]
2016-05-17 22:36:45,528 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:45,528 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:45,529 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:45,529 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...yplatform/ALL_'field'_ALL
2016-05-17 22:36:45,529 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:45,532 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:45,532 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:45,540 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:45,545 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:45,545 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:45,545 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:45,555 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/94876D8DD41D40419794B259EC437BE5/BF553F2DB6B44AF1AABB169233A0A718/.tez/application_1463524605545_0001 doesn't exist and is created
2016-05-17 22:36:45,569 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:45,570 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:45,570 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/94876D8DD41D40419794B259EC437BE5/BF553F2DB6B44AF1AABB169233A0A718/.tez/application_1463524605545_0001
2016-05-17 22:36:45,572 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524605545_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:45,593 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:45,594 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:45,595 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:45,595 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:45,595 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/94876D8DD41D40419794B259EC437BE5/BF553F2DB6B44AF1AABB169233A0A718/.tez/application_1463524605545_0001/localmode-log-dir
2016-05-17 22:36:45,595 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/94876D8DD41D40419794B259EC437BE5/BF553F2DB6B44AF1AABB169233A0A718/.tez/application_1463524605545_0001/localmode-log-dir/history.txt.appattempt_1463524605545_0001_000000, maxErrors=10
2016-05-17 22:36:45,595 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:45,596 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524605545_0001_000000, appSubmitTime=1463524605572, launchTime=1463524605572
2016-05-17 22:36:45,599 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:45,599 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:45,600 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:43650
2016-05-17 22:36:45,601 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:45,601 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:45,602 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:45,604 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524605545_0001_000000, startTime=1463524605604
2016-05-17 22:36:45,604 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:45,604 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:45,670 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:45,670 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:45,670 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/ALL_'field'_ALL, applicationId=application_1463524605545_0001, dagName=[94876D8DD41D40419794B259EC437BE5/BF553F2DB6B44AF1AABB169233A0A718] each/(1/1) ...yplatform/ALL_'field'_ALL
2016-05-17 22:36:45,684 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [94876D8DD41D40419794B259EC437BE5/BF553F2DB6B44AF1AABB169233A0A718] each/(1/1) ...yplatform/ALL_'field'_ALL
2016-05-17 22:36:45,685 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524605545_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/94876D8DD41D40419794B259EC437BE5/BF553F2DB6B44AF1AABB169233A0A718/.tez/application_1463524605545_0001/localmode-log-dir/dag_1463524605545_0001_1.dot
2016-05-17 22:36:45,685 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/94876D8DD41D40419794B259EC437BE5/BF553F2DB6B44AF1AABB169233A0A718/.tez/application_1463524605545_0001/localmode-log-dir/dag_1463524605545_0001_1-tez-dag.pb.txt
2016-05-17 22:36:45,689 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524605545_0001_1
2016-05-17 22:36:45,689 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:45,689 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [94876D8DD41D40419794B259EC437BE5/BF553F2DB6B44AF1AABB169233A0A718] each/(1/1) ...yplatform/ALL_'field'_ALL
2016-05-17 22:36:45 Running Dag: dag_1463524605545_0001_1
2016-05-17 22:36:45,700 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605545_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524605545_0001_1, submitTime=1463524605684
2016-05-17 22:36:45,700 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:45,701 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B70D66E8B5CE4FFFB1D7CDB30D2E7617
2016-05-17 22:36:45,701 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:45,702 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605545_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524605545_0001_1, initTime=1463524605700
2016-05-17 22:36:45,702 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:45,702 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524605545_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:45,703 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605545_0001_1][Event:DAG_STARTED]: dagID=dag_1463524605545_0001_1, startTime=1463524605703
2016-05-17 22:36:45,703 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:45,703 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:45,703 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524605545_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:45,703 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B70D66E8B5CE4FFFB1D7CDB30D2E7617 : {F88E8030F5D7477DAE9C83BFA5DBFED7={InputName=F88E8030F5D7477DAE9C83BFA5DBFED7}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:45,703 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: F88E8030F5D7477DAE9C83BFA5DBFED7, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:45,703 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617]
2016-05-17 22:36:45,703 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617]
2016-05-17 22:36:45,704 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617]
2016-05-17 22:36:45,704 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617]. Starting root input initializers: 1
2016-05-17 22:36:45,704 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/ALL_'field'_ALL, applicationId=application_1463524605545_0001, dagName=[94876D8DD41D40419794B259EC437BE5/BF553F2DB6B44AF1AABB169233A0A718] each/(1/1) ...yplatform/ALL_'field'_ALL
2016-05-17 22:36:45,704 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:45,704 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: F88E8030F5D7477DAE9C83BFA5DBFED7 on vertex vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617]
2016-05-17 22:36:45,706 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:45,706 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input F88E8030F5D7477DAE9C83BFA5DBFED7 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:45,707 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:45,707 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:45,708 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:45,708 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: F88E8030F5D7477DAE9C83BFA5DBFED7 on vertex vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617]
2016-05-17 22:36:45,708 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:45,708 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {F88E8030F5D7477DAE9C83BFA5DBFED7=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:45,709 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617] parallelism set to 1
2016-05-17 22:36:45,709 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:45,709 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605545_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524605545_0001_1_00, eventCount=1
2016-05-17 22:36:45,709 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617]
2016-05-17 22:36:45,709 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=E0C2AD6602EE4B2B998D3C5C03DA53D1, vertexId=vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:45,709 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=E0C2AD6602EE4B2B998D3C5C03DA53D1, vertexId=vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617]
2016-05-17 22:36:45,711 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=E0C2AD6602EE4B2B998D3C5C03DA53D1, vertexName=B70D66E8B5CE4FFFB1D7CDB30D2E7617, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:45,711 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=E0C2AD6602EE4B2B998D3C5C03DA53D1, vertexId=vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617]
2016-05-17 22:36:45,712 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605545_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B70D66E8B5CE4FFFB1D7CDB30D2E7617, vertexId=vertex_1463524605545_0001_1_00, initRequestedTime=1463524605703, initedTime=1463524605712, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:45,712 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617] with distanceFromRoot: 0
2016-05-17 22:36:45,712 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:45,713 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:45,713 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B70D66E8B5CE4FFFB1D7CDB30D2E7617
2016-05-17 22:36:45,713 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617]
2016-05-17 22:36:45,713 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:45,714 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605545_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524605545_0001_1_00, startRequestedTime=1463524605713, startedTime=1463524605713
2016-05-17 22:36:45,714 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:45,714 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:45,715 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605545_0001_1][Event:TASK_STARTED]: vertexName=B70D66E8B5CE4FFFB1D7CDB30D2E7617, taskId=task_1463524605545_0001_1_00_000000, scheduledTime=1463524605714, launchTime=1463524605714
2016-05-17 22:36:45,715 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524605545_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:45,715 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617]
2016-05-17 22:36:45,715 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [94876D8DD41D40419794B259EC437BE5/BF553F2DB6B44AF1AABB169233A0A718] each/(1/1) ...yplatform/ALL_'field'_ALL, VertexName: B70D66E8B5CE4FFFB1D7CDB30D2E7617, VertexParallelism: 1, TaskAttemptID:attempt_1463524605545_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F88E8030F5D7477DAE9C83BFA5DBFED7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E0C2AD6602EE4B2B998D3C5C03DA53D1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:45,715 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524605545_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:45,716 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:45,716 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:45,717 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:45,717 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:45,717 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524605545_0001_00_000001
2016-05-17 22:36:45,717 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524605545_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:45,718 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524605545_0001_00_000001
2016-05-17 22:36:45,718 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:45,719 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524605545_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:45,719 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605545_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524605545_0001_00_000001, launchTime=1463524605719
2016-05-17 22:36:45,719 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:45,720 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524605545_0001_00_000001
2016-05-17 22:36:45,720 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524605545_0001_1_00_000000_0] to container: [container_1463524605545_0001_00_000001]
2016-05-17 22:36:45,721 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524605545_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:45,721 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524605545_0001_1_00_000000_0] started. Is using containerId: [container_1463524605545_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:45,721 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605545_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B70D66E8B5CE4FFFB1D7CDB30D2E7617, taskAttemptId=attempt_1463524605545_0001_1_00_000000_0, startTime=1463524605721, containerId=container_1463524605545_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524605545_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:45,721 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524605545_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:45,721 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:45,721 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524605545_0001_00_000001 given task: attempt_1463524605545_0001_1_00_000000_0
2016-05-17 22:36:45,721 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524605545_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524605545_0001_1_00_000000_0
2016-05-17 22:36:45,721 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524605545_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:45 Starting to run new task attempt: attempt_1463524605545_0001_1_00_000000_0
2016-05-17 22:36:45,722 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524605545_0001_1_00_000000_0
2016-05-17 22:36:45,722 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:45,722 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524605545_0001, with dag id: dag_1463524605545_0001_1
2016-05-17 22:36:45,722 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:45,722 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:45,722 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:45,722 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:45,722 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:45,722 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [94876D8DD41D40419794B259EC437BE5/BF553F2DB6B44AF1AABB169233A0A718] each/(1/1) ...yplatform/ALL_'field'_ALL, VertexName: B70D66E8B5CE4FFFB1D7CDB30D2E7617, VertexParallelism: 1, TaskAttemptID:attempt_1463524605545_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F88E8030F5D7477DAE9C83BFA5DBFED7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E0C2AD6602EE4B2B998D3C5C03DA53D1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:45,723 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:45,723 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524605545_0001_1_00_000000_0
2016-05-17 22:36:45,723 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:45,723 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:45,724 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524605545_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:45,724 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F88E8030F5D7477DAE9C83BFA5DBFED7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:45,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:45,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:45,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F88E8030F5D7477DAE9C83BFA5DBFED7
2016-05-17 22:36:45,725 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: F88E8030F5D7477DAE9C83BFA5DBFED7
2016-05-17 22:36:45,727 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:45,728 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:45,728 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: F88E8030F5D7477DAE9C83BFA5DBFED7
2016-05-17 22:36:45,728 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F88E8030F5D7477DAE9C83BFA5DBFED7
2016-05-17 22:36:45,729 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E0C2AD6602EE4B2B998D3C5C03DA53D1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:45,729 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:45,729 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E0C2AD6602EE4B2B998D3C5C03DA53D1
2016-05-17 22:36:45,729 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:45,729 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: E0C2AD6602EE4B2B998D3C5C03DA53D1
2016-05-17 22:36:45,733 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B70D66E8B5CE4FFFB1D7CDB30D2E7617, ordinal: 0
2016-05-17 22:36:45,733 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B70D66E8B5CE4FFFB1D7CDB30D2E7617, mem on start (mb), free: 241, total: 337, max: 672
2016-05-17 22:36:45,733 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:45,733 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:45,733 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:45,738 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:45,738 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E0C2AD6602EE4B2B998D3C5C03DA53D1
2016-05-17 22:36:45,738 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:45,738 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:45,738 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:45,738 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:45,738 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:45,738 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:45,738 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:45,739 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:45,739 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F88E8030F5D7477DAE9C83BFA5DBFED7, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:45,739 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E0C2AD6602EE4B2B998D3C5C03DA53D1, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:45,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:45,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F88E8030F5D7477DAE9C83BFA5DBFED7 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:45,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F88E8030F5D7477DAE9C83BFA5DBFED7
2016-05-17 22:36:45,739 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:45,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F88E8030F5D7477DAE9C83BFA5DBFED7
2016-05-17 22:36:45,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:45,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:45,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:45,740 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524605545_0001_1_00_000000_0
2016-05-17 22:36:45,741 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:45,742 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6bc79208
2016-05-17 22:36:45,743 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6bc79208
2016-05-17 22:36:45,743 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:45,743 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:45,743 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:45,743 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: F88E8030F5D7477DAE9C83BFA5DBFED7
2016-05-17 22:36:45,743 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_ALL"], id: E0C2AD6602EE4B2B998D3C5C03DA53D1
2016-05-17 22:36:45,744 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_ALL"]
2016-05-17 22:36:45,744 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:45,744 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:45,744 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B70D66E8B5CE4FFFB1D7CDB30D2E7617, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:45,747 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:45,747 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:45,747 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524605545_0001_1_00_000000_0
2016-05-17 22:36:45,748 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524605545_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:45,748 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524605545_0001_1_00_000000_0
2016-05-17 22:36:45,748 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B70D66E8B5CE4FFFB1D7CDB30D2E7617, mem on close (mb), free: 239, total: 337, max: 672
2016-05-17 22:36:45 Completed running task attempt: attempt_1463524605545_0001_1_00_000000_0
2016-05-17 22:36:45,749 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524605545_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:45,749 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=353370112, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524605744, Process_Duration=4, Process_End_Time=1463524605748, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:45,749 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:45,749 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:45,749 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605545_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B70D66E8B5CE4FFFB1D7CDB30D2E7617, taskAttemptId=attempt_1463524605545_0001_1_00_000000_0, startTime=1463524605721, finishTime=1463524605749, timeTaken=28, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=353370112, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524605744, Process_Duration=4, Process_End_Time=1463524605748, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:45,750 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524605545_0001_00_000001
2016-05-17 22:36:45,750 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524605545_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:45,749 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:45,750 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524605545_0001_1_00_000000_0
2016-05-17 22:36:45,750 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:45,750 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:45,750 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605545_0001_1][Event:TASK_FINISHED]: vertexName=B70D66E8B5CE4FFFB1D7CDB30D2E7617, taskId=task_1463524605545_0001_1_00_000000, startTime=1463524605721, finishTime=1463524605750, timeTaken=29, status=SUCCEEDED, successfulAttemptID=attempt_1463524605545_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=353370112, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524605744, Process_Duration=4, Process_End_Time=1463524605748, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:45,751 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524605545_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:45,751 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617] : 1
2016-05-17 22:36:45,751 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617]
2016-05-17 22:36:45,752 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:45,752 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605545_0001_1][Event:VERTEX_FINISHED]: vertexName=B70D66E8B5CE4FFFB1D7CDB30D2E7617, vertexId=vertex_1463524605545_0001_1_00, initRequestedTime=1463524605703, initedTime=1463524605712, startRequestedTime=1463524605713, startedTime=1463524605713, finishTime=1463524605751, timeTaken=38, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=353370112, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524605744, Process_Duration=4, Process_End_Time=1463524605748, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524605721, firstTasksToStart=[ task_1463524605545_0001_1_00_000000 ], lastTaskFinishTime=1463524605749, lastTasksToFinish=[ task_1463524605545_0001_1_00_000000 ], minTaskDuration=28, maxTaskDuration=28, avgTaskDuration=28.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524605545_0001_1_00_000000 ], longestDurationTasks=[ task_1463524605545_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:45,752 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:45,752 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524605545_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:45,752 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605545_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524605545_0001_00_000001, stoppedTime=1463524605752, exitStatus=0
2016-05-17 22:36:45,752 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:45,752 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524605545_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:45,752 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:45,752 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524605545_0001_1
2016-05-17 22:36:45,753 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605545_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524605545_0001_1
2016-05-17 22:36:45,753 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: E0C2AD6602EE4B2B998D3C5C03DA53D1 for vertex: vertex_1463524605545_0001_1_00 [B70D66E8B5CE4FFFB1D7CDB30D2E7617]
2016-05-17 22:36:45,760 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524605545_0001_1, queueSize=0
2016-05-17 22:36:45,760 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:45,760 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605545_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524605545_0001_1, startTime=1463524605703, finishTime=1463524605759, timeTaken=56, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=353370112, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524605744, Process_Duration=4, Process_End_Time=1463524605748, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:45,760 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524605545_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:45,761 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524605545_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:45,761 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524605545_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:36:45,761 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524605545_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:45,761 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524605545_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:45,761 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524605545_0001_00_000001, Exiting
2016-05-17 22:36:45,762 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524605545_0001_00_000001
2016-05-17 22:36:45,762 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524605545_0001_00_000001
2016-05-17 22:36:45 Completed Dag: dag_1463524605545_0001_1
2016-05-17 22:36:45,762 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524605545_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:45,762 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524605545_0001_1_post
2016-05-17 22:36:45,762 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:45,762 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:45,762 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524605545_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:36:45,763 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524605545_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:45,763 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524605545_0001_00_000001 completed successfully
2016-05-17 22:36:45,764 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524605545_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:36:45,766 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/ALL_'field'_ALL, applicationId=application_1463524605545_0001
2016-05-17 22:36:45,768 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:45,768 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:45,768 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:45,768 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:45,768 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:45,769 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:45,769 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:45,769 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:45,769 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:45,769 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:45,769 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:45,770 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:45,770 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:45,773 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/94876D8DD41D40419794B259EC437BE5/BF553F2DB6B44AF1AABB169233A0A718/.tez/application_1463524605545_0001
2016-05-17 22:36:45,774 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_ALL/_temporary
2016-05-17 22:36:45,774 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.246
2016-05-17 22:36:45,838 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_'field'_ALL
2016-05-17 22:36:45,859 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:45,859 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:45,883 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.002
2016-05-17 22:36:45,885 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-05-17 22:36:45,885 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:45,885 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:45,885 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_'field'_REPLACE
2016-05-17 22:36:45,906 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:45,906 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:45,935 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 22:36:45,937 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:45,937 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:45,944 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-17 22:36:45,944 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:45,944 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:45,945 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:45,957 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:45,957 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_SWAP
2016-05-17 22:36:45,966 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:45,966 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:45,966 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_SWAP"]
2016-05-17 22:36:45,966 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:45,967 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:45,967 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:45,967 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...platform/ALL_'field'_SWAP
2016-05-17 22:36:45,967 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:45,978 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:45,983 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:45,983 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:45,983 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:45,992 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CFCC51C7526A4D168C2A7416E883F070/CC200ADEB074481DA92E7CABA221B227/.tez/application_1463524605983_0001 doesn't exist and is created
2016-05-17 22:36:46,007 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:46,007 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:46,008 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CFCC51C7526A4D168C2A7416E883F070/CC200ADEB074481DA92E7CABA221B227/.tez/application_1463524605983_0001
2016-05-17 22:36:46,010 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524605983_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:46,023 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:46,023 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:46,024 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:46,024 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:46,024 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CFCC51C7526A4D168C2A7416E883F070/CC200ADEB074481DA92E7CABA221B227/.tez/application_1463524605983_0001/localmode-log-dir
2016-05-17 22:36:46,025 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CFCC51C7526A4D168C2A7416E883F070/CC200ADEB074481DA92E7CABA221B227/.tez/application_1463524605983_0001/localmode-log-dir/history.txt.appattempt_1463524605983_0001_000000, maxErrors=10
2016-05-17 22:36:46,025 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:46,025 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524605983_0001_000000, appSubmitTime=1463524606010, launchTime=1463524606010
2016-05-17 22:36:46,028 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:46,028 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:46,029 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:46,030 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:46,031 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:56974
2016-05-17 22:36:46,032 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:46,033 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524605983_0001_000000, startTime=1463524606033
2016-05-17 22:36:46,033 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:46,033 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:46,107 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:46,108 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:46,108 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/ALL_'field'_SWAP, applicationId=application_1463524605983_0001, dagName=[CFCC51C7526A4D168C2A7416E883F070/CC200ADEB074481DA92E7CABA221B227] each/(1/1) ...platform/ALL_'field'_SWAP
2016-05-17 22:36:46,122 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [CFCC51C7526A4D168C2A7416E883F070/CC200ADEB074481DA92E7CABA221B227] each/(1/1) ...platform/ALL_'field'_SWAP
2016-05-17 22:36:46,124 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524605983_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CFCC51C7526A4D168C2A7416E883F070/CC200ADEB074481DA92E7CABA221B227/.tez/application_1463524605983_0001/localmode-log-dir/dag_1463524605983_0001_1.dot
2016-05-17 22:36:46,124 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CFCC51C7526A4D168C2A7416E883F070/CC200ADEB074481DA92E7CABA221B227/.tez/application_1463524605983_0001/localmode-log-dir/dag_1463524605983_0001_1-tez-dag.pb.txt
2016-05-17 22:36:46,128 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524605983_0001_1
2016-05-17 22:36:46,128 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:46,128 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [CFCC51C7526A4D168C2A7416E883F070/CC200ADEB074481DA92E7CABA221B227] each/(1/1) ...platform/ALL_'field'_SWAP
2016-05-17 22:36:46 Running Dag: dag_1463524605983_0001_1
2016-05-17 22:36:46,140 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605983_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524605983_0001_1, submitTime=1463524606123
2016-05-17 22:36:46,140 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:46,141 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B722C8C39A184F069A9999D0ED94D53B
2016-05-17 22:36:46,141 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:46,141 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:46,141 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605983_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524605983_0001_1, initTime=1463524606140
2016-05-17 22:36:46,142 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524605983_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:46,142 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605983_0001_1][Event:DAG_STARTED]: dagID=dag_1463524605983_0001_1, startTime=1463524606142
2016-05-17 22:36:46,142 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:46,142 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524605983_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:46,142 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/ALL_'field'_SWAP, applicationId=application_1463524605983_0001, dagName=[CFCC51C7526A4D168C2A7416E883F070/CC200ADEB074481DA92E7CABA221B227] each/(1/1) ...platform/ALL_'field'_SWAP
2016-05-17 22:36:46,142 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B722C8C39A184F069A9999D0ED94D53B : {2A3B0EE51CA0471D80D60EE32592352A={InputName=2A3B0EE51CA0471D80D60EE32592352A}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:46,142 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 2A3B0EE51CA0471D80D60EE32592352A, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:46,142 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B]
2016-05-17 22:36:46,142 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:46,143 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B]
2016-05-17 22:36:46,143 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B]
2016-05-17 22:36:46,143 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B]. Starting root input initializers: 1
2016-05-17 22:36:46,143 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:46,143 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 2A3B0EE51CA0471D80D60EE32592352A on vertex vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B]
2016-05-17 22:36:46,146 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:46,147 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 2A3B0EE51CA0471D80D60EE32592352A asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:46,147 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:46,148 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:46,149 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:46,149 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 2A3B0EE51CA0471D80D60EE32592352A on vertex vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B]
2016-05-17 22:36:46,149 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:46,150 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {2A3B0EE51CA0471D80D60EE32592352A=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:46,150 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B] parallelism set to 1
2016-05-17 22:36:46,150 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:46,150 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605983_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524605983_0001_1_00, eventCount=1
2016-05-17 22:36:46,150 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B]
2016-05-17 22:36:46,150 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=8677A1986CDC4DC1951C79AF6C6D9125, vertexId=vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:46,150 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=8677A1986CDC4DC1951C79AF6C6D9125, vertexId=vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B]
2016-05-17 22:36:46,152 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=8677A1986CDC4DC1951C79AF6C6D9125, vertexName=B722C8C39A184F069A9999D0ED94D53B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:46,152 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=8677A1986CDC4DC1951C79AF6C6D9125, vertexId=vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B]
2016-05-17 22:36:46,153 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605983_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B722C8C39A184F069A9999D0ED94D53B, vertexId=vertex_1463524605983_0001_1_00, initRequestedTime=1463524606142, initedTime=1463524606153, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:46,153 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B] with distanceFromRoot: 0
2016-05-17 22:36:46,153 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:46,153 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:46,154 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B722C8C39A184F069A9999D0ED94D53B
2016-05-17 22:36:46,154 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B]
2016-05-17 22:36:46,154 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605983_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524605983_0001_1_00, startRequestedTime=1463524606154, startedTime=1463524606154
2016-05-17 22:36:46,154 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:46,154 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:46,155 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:46,155 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605983_0001_1][Event:TASK_STARTED]: vertexName=B722C8C39A184F069A9999D0ED94D53B, taskId=task_1463524605983_0001_1_00_000000, scheduledTime=1463524606155, launchTime=1463524606155
2016-05-17 22:36:46,155 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524605983_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:46,155 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B]
2016-05-17 22:36:46,155 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [CFCC51C7526A4D168C2A7416E883F070/CC200ADEB074481DA92E7CABA221B227] each/(1/1) ...platform/ALL_'field'_SWAP, VertexName: B722C8C39A184F069A9999D0ED94D53B, VertexParallelism: 1, TaskAttemptID:attempt_1463524605983_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2A3B0EE51CA0471D80D60EE32592352A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=8677A1986CDC4DC1951C79AF6C6D9125, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:46,155 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524605983_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:46,155 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:46,156 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:46,156 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:46,156 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:46,157 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524605983_0001_00_000001
2016-05-17 22:36:46,157 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524605983_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:46,158 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524605983_0001, with dag id: dag_1463524605983_0001_1
2016-05-17 22:36:46,158 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524605983_0001_00_000001
2016-05-17 22:36:46,158 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:46,159 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524605983_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:46,159 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:46,160 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605983_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524605983_0001_00_000001, launchTime=1463524606159
2016-05-17 22:36:46,160 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524605983_0001_00_000001
2016-05-17 22:36:46,161 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524605983_0001_1_00_000000_0] to container: [container_1463524605983_0001_00_000001]
2016-05-17 22:36:46,161 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524605983_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:46,161 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524605983_0001_1_00_000000_0] started. Is using containerId: [container_1463524605983_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:46,161 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605983_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B722C8C39A184F069A9999D0ED94D53B, taskAttemptId=attempt_1463524605983_0001_1_00_000000_0, startTime=1463524606161, containerId=container_1463524605983_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524605983_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:46,162 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524605983_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:46,162 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524605983_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:46,162 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:46,162 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524605983_0001_00_000001 given task: attempt_1463524605983_0001_1_00_000000_0
2016-05-17 22:36:46,162 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524605983_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524605983_0001_1_00_000000_0
2016-05-17 22:36:46 Starting to run new task attempt: attempt_1463524605983_0001_1_00_000000_0
2016-05-17 22:36:46,163 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524605983_0001_1_00_000000_0
2016-05-17 22:36:46,163 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:46,163 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:46,163 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:46,163 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:46,163 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:46,163 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:46,163 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [CFCC51C7526A4D168C2A7416E883F070/CC200ADEB074481DA92E7CABA221B227] each/(1/1) ...platform/ALL_'field'_SWAP, VertexName: B722C8C39A184F069A9999D0ED94D53B, VertexParallelism: 1, TaskAttemptID:attempt_1463524605983_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2A3B0EE51CA0471D80D60EE32592352A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=8677A1986CDC4DC1951C79AF6C6D9125, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:46,163 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:46,164 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524605983_0001_1_00_000000_0
2016-05-17 22:36:46,164 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:46,164 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:46,164 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=2A3B0EE51CA0471D80D60EE32592352A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:46,165 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524605983_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:46,165 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:46,165 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:46,165 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 2A3B0EE51CA0471D80D60EE32592352A
2016-05-17 22:36:46,165 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 2A3B0EE51CA0471D80D60EE32592352A
2016-05-17 22:36:46,166 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=8677A1986CDC4DC1951C79AF6C6D9125, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:46,166 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:46,166 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 8677A1986CDC4DC1951C79AF6C6D9125
2016-05-17 22:36:46,167 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:46,167 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 8677A1986CDC4DC1951C79AF6C6D9125
2016-05-17 22:36:46,167 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:46,168 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:46,168 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 2A3B0EE51CA0471D80D60EE32592352A
2016-05-17 22:36:46,168 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 2A3B0EE51CA0471D80D60EE32592352A
2016-05-17 22:36:46,172 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B722C8C39A184F069A9999D0ED94D53B, ordinal: 0
2016-05-17 22:36:46,172 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B722C8C39A184F069A9999D0ED94D53B, mem on start (mb), free: 150, total: 337, max: 672
2016-05-17 22:36:46,172 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:46,172 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:46,172 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:46,177 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:46,177 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 8677A1986CDC4DC1951C79AF6C6D9125
2016-05-17 22:36:46,177 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:46,177 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:46,177 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:46,177 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:46,178 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:46,178 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:46,178 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:46,178 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:46,178 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 2A3B0EE51CA0471D80D60EE32592352A, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:46,178 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 8677A1986CDC4DC1951C79AF6C6D9125, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:46,178 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:46,178 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 2A3B0EE51CA0471D80D60EE32592352A being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:46,178 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 2A3B0EE51CA0471D80D60EE32592352A
2016-05-17 22:36:46,178 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:46,178 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 2A3B0EE51CA0471D80D60EE32592352A
2016-05-17 22:36:46,178 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:46,178 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:46,178 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:46,179 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524605983_0001_1_00_000000_0
2016-05-17 22:36:46,179 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:46,179 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@a26cd3b
2016-05-17 22:36:46,180 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@a26cd3b
2016-05-17 22:36:46,180 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:46,181 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:46,182 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:46,183 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 2A3B0EE51CA0471D80D60EE32592352A
2016-05-17 22:36:46,183 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_SWAP"], id: 8677A1986CDC4DC1951C79AF6C6D9125
2016-05-17 22:36:46,183 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_SWAP"]
2016-05-17 22:36:46,183 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:46,183 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:46,183 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B722C8C39A184F069A9999D0ED94D53B, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:46,186 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:46,186 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:46,187 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524605983_0001_1_00_000000_0
2016-05-17 22:36:46,187 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524605983_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:46,187 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524605983_0001_1_00_000000_0
2016-05-17 22:36:46,187 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B722C8C39A184F069A9999D0ED94D53B, mem on close (mb), free: 148, total: 337, max: 672
2016-05-17 22:36:46 Completed running task attempt: attempt_1463524605983_0001_1_00_000000_0
2016-05-17 22:36:46,188 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524605983_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:46,189 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=353370112, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524606183, Process_Duration=4, Process_End_Time=1463524606187, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:46,189 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605983_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B722C8C39A184F069A9999D0ED94D53B, taskAttemptId=attempt_1463524605983_0001_1_00_000000_0, startTime=1463524606161, finishTime=1463524606189, timeTaken=28, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=353370112, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524606183, Process_Duration=4, Process_End_Time=1463524606187, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:46,189 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524605983_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:46,189 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:46,189 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524605983_0001_1_00_000000_0
2016-05-17 22:36:46,190 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:46,190 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605983_0001_1][Event:TASK_FINISHED]: vertexName=B722C8C39A184F069A9999D0ED94D53B, taskId=task_1463524605983_0001_1_00_000000, startTime=1463524606161, finishTime=1463524606189, timeTaken=28, status=SUCCEEDED, successfulAttemptID=attempt_1463524605983_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=353370112, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524606183, Process_Duration=4, Process_End_Time=1463524606187, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:46,190 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:46,190 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524605983_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:46,190 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B] : 1
2016-05-17 22:36:46,190 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B]
2016-05-17 22:36:46,190 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:46,191 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:46,191 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524605983_0001_00_000001
2016-05-17 22:36:46,191 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:46,191 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605983_0001_1][Event:VERTEX_FINISHED]: vertexName=B722C8C39A184F069A9999D0ED94D53B, vertexId=vertex_1463524605983_0001_1_00, initRequestedTime=1463524606142, initedTime=1463524606153, startRequestedTime=1463524606154, startedTime=1463524606154, finishTime=1463524606190, timeTaken=36, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=353370112, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524606183, Process_Duration=4, Process_End_Time=1463524606187, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524606161, firstTasksToStart=[ task_1463524605983_0001_1_00_000000 ], lastTaskFinishTime=1463524606189, lastTasksToFinish=[ task_1463524605983_0001_1_00_000000 ], minTaskDuration=28, maxTaskDuration=28, avgTaskDuration=28.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524605983_0001_1_00_000000 ], longestDurationTasks=[ task_1463524605983_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:46,191 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:46,191 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524605983_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:46,192 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605983_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524605983_0001_00_000001, stoppedTime=1463524606191, exitStatus=0
2016-05-17 22:36:46,192 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:46,192 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524605983_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:46,192 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:46,192 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524605983_0001_1
2016-05-17 22:36:46,192 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605983_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524605983_0001_1
2016-05-17 22:36:46,192 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 8677A1986CDC4DC1951C79AF6C6D9125 for vertex: vertex_1463524605983_0001_1_00 [B722C8C39A184F069A9999D0ED94D53B]
2016-05-17 22:36:46,201 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524605983_0001_1, queueSize=0
2016-05-17 22:36:46,201 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:46,201 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524605983_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:46,201 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524605983_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:46,201 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524605983_0001_00_000001, Exiting
2016-05-17 22:36:46,201 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524605983_0001_00_000001
2016-05-17 22:36:46,201 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524605983_0001_00_000001
2016-05-17 22:36:46,202 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524605983_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:46,202 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524605983_0001_00_000001 completed successfully
2016-05-17 22:36:46,202 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524605983_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524605983_0001_1, startTime=1463524606142, finishTime=1463524606200, timeTaken=58, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=353370112, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524606183, Process_Duration=4, Process_End_Time=1463524606187, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:46,203 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524605983_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:46,203 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524605983_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:46,203 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524605983_0001_00_000001
2016-05-17 22:36:46,204 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524605983_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:46 Completed Dag: dag_1463524605983_0001_1
2016-05-17 22:36:46,205 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524605983_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:46,205 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524605983_0001_1_post
2016-05-17 22:36:46,205 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:46,206 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:46,209 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/ALL_'field'_SWAP, applicationId=application_1463524605983_0001
2016-05-17 22:36:46,210 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:46,210 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:46,211 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:46,211 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:46,211 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:46,211 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:46,211 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:46,211 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:46,212 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:46,212 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:46,212 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:46,212 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:46,212 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:46,215 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CFCC51C7526A4D168C2A7416E883F070/CC200ADEB074481DA92E7CABA221B227/.tez/application_1463524605983_0001
2016-05-17 22:36:46,215 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_'field'_SWAP/_temporary
2016-05-17 22:36:46,215 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.249
2016-05-17 22:36:46,278 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_'field'_SWAP
2016-05-17 22:36:46,299 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:46,299 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:46,329 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 22:36:46,332 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 22:36:46,332 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:46,332 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:46,333 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:46,341 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:46,341 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:46,346 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:46,347 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_-1
2016-05-17 22:36:46,356 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:46,356 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:46,356 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_-1"]
2016-05-17 22:36:46,356 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:46,356 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:46,356 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:46,356 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...lyplatform/ALL_UNKNOWN_-1
2016-05-17 22:36:46,356 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:46,367 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:46,371 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:46,372 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:46,372 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:46,381 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/416143C490E8446DA6A7AD56617C9F20/B1931B653B6D4F4F8D941D85610A4409/.tez/application_1463524606372_0001 doesn't exist and is created
2016-05-17 22:36:46,396 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:46,396 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:46,396 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/416143C490E8446DA6A7AD56617C9F20/B1931B653B6D4F4F8D941D85610A4409/.tez/application_1463524606372_0001
2016-05-17 22:36:46,398 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524606372_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:46,407 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:46,408 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:46,409 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:46,409 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:46,409 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/416143C490E8446DA6A7AD56617C9F20/B1931B653B6D4F4F8D941D85610A4409/.tez/application_1463524606372_0001/localmode-log-dir
2016-05-17 22:36:46,409 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/416143C490E8446DA6A7AD56617C9F20/B1931B653B6D4F4F8D941D85610A4409/.tez/application_1463524606372_0001/localmode-log-dir/history.txt.appattempt_1463524606372_0001_000000, maxErrors=10
2016-05-17 22:36:46,409 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:46,409 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524606372_0001_000000, appSubmitTime=1463524606398, launchTime=1463524606398
2016-05-17 22:36:46,424 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:46,424 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:46,425 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:46,425 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:46,427 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:55608
2016-05-17 22:36:46,428 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:46,429 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524606372_0001_000000, startTime=1463524606429
2016-05-17 22:36:46,429 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:46,429 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:46,496 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:46,496 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:46,496 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_UNKNOWN_-1, applicationId=application_1463524606372_0001, dagName=[416143C490E8446DA6A7AD56617C9F20/B1931B653B6D4F4F8D941D85610A4409] each/(1/1) ...lyplatform/ALL_UNKNOWN_-1
2016-05-17 22:36:46,511 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [416143C490E8446DA6A7AD56617C9F20/B1931B653B6D4F4F8D941D85610A4409] each/(1/1) ...lyplatform/ALL_UNKNOWN_-1
2016-05-17 22:36:46,512 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524606372_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/416143C490E8446DA6A7AD56617C9F20/B1931B653B6D4F4F8D941D85610A4409/.tez/application_1463524606372_0001/localmode-log-dir/dag_1463524606372_0001_1.dot
2016-05-17 22:36:46,512 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/416143C490E8446DA6A7AD56617C9F20/B1931B653B6D4F4F8D941D85610A4409/.tez/application_1463524606372_0001/localmode-log-dir/dag_1463524606372_0001_1-tez-dag.pb.txt
2016-05-17 22:36:46,517 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524606372_0001_1
2016-05-17 22:36:46,517 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:46,517 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [416143C490E8446DA6A7AD56617C9F20/B1931B653B6D4F4F8D941D85610A4409] each/(1/1) ...lyplatform/ALL_UNKNOWN_-1
2016-05-17 22:36:46 Running Dag: dag_1463524606372_0001_1
2016-05-17 22:36:46,528 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606372_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524606372_0001_1, submitTime=1463524606511
2016-05-17 22:36:46,528 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:46,529 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B026AAE895E44B6094B24DA6F3E436D4
2016-05-17 22:36:46,529 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:46,529 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606372_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524606372_0001_1, initTime=1463524606528
2016-05-17 22:36:46,529 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524606372_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:46,529 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606372_0001_1][Event:DAG_STARTED]: dagID=dag_1463524606372_0001_1, startTime=1463524606529
2016-05-17 22:36:46,530 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:46,530 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524606372_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:46,530 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B026AAE895E44B6094B24DA6F3E436D4 : {839CB102ACD240B7868E0AE68CF60A27={InputName=839CB102ACD240B7868E0AE68CF60A27}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:46,530 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 839CB102ACD240B7868E0AE68CF60A27, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:46,530 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4]
2016-05-17 22:36:46,530 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4]
2016-05-17 22:36:46,530 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4]
2016-05-17 22:36:46,530 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:46,530 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4]. Starting root input initializers: 1
2016-05-17 22:36:46,531 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:46,530 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_UNKNOWN_-1, applicationId=application_1463524606372_0001, dagName=[416143C490E8446DA6A7AD56617C9F20/B1931B653B6D4F4F8D941D85610A4409] each/(1/1) ...lyplatform/ALL_UNKNOWN_-1
2016-05-17 22:36:46,531 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:46,532 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 839CB102ACD240B7868E0AE68CF60A27 on vertex vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4]
2016-05-17 22:36:46,533 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:46,534 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 839CB102ACD240B7868E0AE68CF60A27 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:46,534 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:46,534 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:46,535 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:46,536 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 839CB102ACD240B7868E0AE68CF60A27 on vertex vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4]
2016-05-17 22:36:46,536 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:46,536 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {839CB102ACD240B7868E0AE68CF60A27=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:46,536 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4] parallelism set to 1
2016-05-17 22:36:46,536 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:46,536 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606372_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524606372_0001_1_00, eventCount=1
2016-05-17 22:36:46,537 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4]
2016-05-17 22:36:46,537 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=5771B51EBAB6446587C731C3BBB5E9B6, vertexId=vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:46,537 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=5771B51EBAB6446587C731C3BBB5E9B6, vertexId=vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4]
2016-05-17 22:36:46,539 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=5771B51EBAB6446587C731C3BBB5E9B6, vertexName=B026AAE895E44B6094B24DA6F3E436D4, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:46,539 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=5771B51EBAB6446587C731C3BBB5E9B6, vertexId=vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4]
2016-05-17 22:36:46,540 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:46,540 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606372_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B026AAE895E44B6094B24DA6F3E436D4, vertexId=vertex_1463524606372_0001_1_00, initRequestedTime=1463524606530, initedTime=1463524606540, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:46,540 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4] with distanceFromRoot: 0
2016-05-17 22:36:46,540 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:46,541 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B026AAE895E44B6094B24DA6F3E436D4
2016-05-17 22:36:46,541 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4]
2016-05-17 22:36:46,541 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:46,541 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606372_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524606372_0001_1_00, startRequestedTime=1463524606540, startedTime=1463524606540
2016-05-17 22:36:46,541 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:46,542 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:46,542 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606372_0001_1][Event:TASK_STARTED]: vertexName=B026AAE895E44B6094B24DA6F3E436D4, taskId=task_1463524606372_0001_1_00_000000, scheduledTime=1463524606542, launchTime=1463524606542
2016-05-17 22:36:46,542 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524606372_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:46,542 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4]
2016-05-17 22:36:46,543 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [416143C490E8446DA6A7AD56617C9F20/B1931B653B6D4F4F8D941D85610A4409] each/(1/1) ...lyplatform/ALL_UNKNOWN_-1, VertexName: B026AAE895E44B6094B24DA6F3E436D4, VertexParallelism: 1, TaskAttemptID:attempt_1463524606372_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=839CB102ACD240B7868E0AE68CF60A27, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=5771B51EBAB6446587C731C3BBB5E9B6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:46,543 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524606372_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:46,543 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:46,544 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:46,544 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:46,544 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:46,545 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524606372_0001_00_000001
2016-05-17 22:36:46,545 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524606372_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:46,546 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524606372_0001_00_000001
2016-05-17 22:36:46,546 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:46,546 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524606372_0001, with dag id: dag_1463524606372_0001_1
2016-05-17 22:36:46,547 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524606372_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:46,547 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:46,547 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606372_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524606372_0001_00_000001, launchTime=1463524606547
2016-05-17 22:36:46,549 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524606372_0001_00_000001
2016-05-17 22:36:46,549 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524606372_0001_1_00_000000_0] to container: [container_1463524606372_0001_00_000001]
2016-05-17 22:36:46,550 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524606372_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:46,550 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524606372_0001_00_000001 given task: attempt_1463524606372_0001_1_00_000000_0
2016-05-17 22:36:46,550 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524606372_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524606372_0001_1_00_000000_0
2016-05-17 22:36:46,550 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524606372_0001_1_00_000000_0] started. Is using containerId: [container_1463524606372_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:46,550 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606372_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B026AAE895E44B6094B24DA6F3E436D4, taskAttemptId=attempt_1463524606372_0001_1_00_000000_0, startTime=1463524606550, containerId=container_1463524606372_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524606372_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:46,550 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524606372_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:46 Starting to run new task attempt: attempt_1463524606372_0001_1_00_000000_0
2016-05-17 22:36:46,551 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524606372_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:46,551 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524606372_0001_1_00_000000_0
2016-05-17 22:36:46,551 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:46,551 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:46,551 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:46,551 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:46,551 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:46,551 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:46,552 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:46,552 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [416143C490E8446DA6A7AD56617C9F20/B1931B653B6D4F4F8D941D85610A4409] each/(1/1) ...lyplatform/ALL_UNKNOWN_-1, VertexName: B026AAE895E44B6094B24DA6F3E436D4, VertexParallelism: 1, TaskAttemptID:attempt_1463524606372_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=839CB102ACD240B7868E0AE68CF60A27, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=5771B51EBAB6446587C731C3BBB5E9B6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:46,552 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:46,552 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524606372_0001_1_00_000000_0
2016-05-17 22:36:46,552 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:46,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:46,553 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524606372_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:46,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:46,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=5771B51EBAB6446587C731C3BBB5E9B6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:46,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:46,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 5771B51EBAB6446587C731C3BBB5E9B6
2016-05-17 22:36:46,554 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:46,554 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 5771B51EBAB6446587C731C3BBB5E9B6
2016-05-17 22:36:46,555 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:46,560 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B026AAE895E44B6094B24DA6F3E436D4, ordinal: 0
2016-05-17 22:36:46,560 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B026AAE895E44B6094B24DA6F3E436D4, mem on start (mb), free: 233, total: 328, max: 672
2016-05-17 22:36:46,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:46,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:46,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=839CB102ACD240B7868E0AE68CF60A27, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:46,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:46,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 839CB102ACD240B7868E0AE68CF60A27
2016-05-17 22:36:46,561 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 839CB102ACD240B7868E0AE68CF60A27
2016-05-17 22:36:46,563 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:46,563 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 839CB102ACD240B7868E0AE68CF60A27
2016-05-17 22:36:46,563 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 839CB102ACD240B7868E0AE68CF60A27
2016-05-17 22:36:46,564 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:46,565 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:46,565 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 5771B51EBAB6446587C731C3BBB5E9B6
2016-05-17 22:36:46,566 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:46,566 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:46,566 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:46,566 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:46,566 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:46,567 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:46,567 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:46,567 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:46,567 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 5771B51EBAB6446587C731C3BBB5E9B6, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:46,567 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 839CB102ACD240B7868E0AE68CF60A27, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:46,567 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:46,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 839CB102ACD240B7868E0AE68CF60A27 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:46,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:46,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:46,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 839CB102ACD240B7868E0AE68CF60A27
2016-05-17 22:36:46,568 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:46,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 839CB102ACD240B7868E0AE68CF60A27
2016-05-17 22:36:46,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:46,569 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524606372_0001_1_00_000000_0
2016-05-17 22:36:46,569 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:46,569 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@17316832
2016-05-17 22:36:46,570 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@17316832
2016-05-17 22:36:46,570 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:46,570 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:46,571 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:46,572 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 839CB102ACD240B7868E0AE68CF60A27
2016-05-17 22:36:46,572 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_-1"], id: 5771B51EBAB6446587C731C3BBB5E9B6
2016-05-17 22:36:46,572 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_-1"]
2016-05-17 22:36:46,573 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:46,573 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:46,573 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B026AAE895E44B6094B24DA6F3E436D4, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:46,576 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:46,576 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:46,577 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524606372_0001_1_00_000000_0
2016-05-17 22:36:46,577 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524606372_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:46,578 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524606372_0001_1_00_000000_0
2016-05-17 22:36:46,578 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B026AAE895E44B6094B24DA6F3E436D4, mem on close (mb), free: 231, total: 328, max: 672
2016-05-17 22:36:46 Completed running task attempt: attempt_1463524606372_0001_1_00_000000_0
2016-05-17 22:36:46,579 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524606372_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:46,579 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=343932928, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524606573, Process_Duration=5, Process_End_Time=1463524606578, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:46,579 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:46,579 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606372_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B026AAE895E44B6094B24DA6F3E436D4, taskAttemptId=attempt_1463524606372_0001_1_00_000000_0, startTime=1463524606550, finishTime=1463524606579, timeTaken=29, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=343932928, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524606573, Process_Duration=5, Process_End_Time=1463524606578, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:46,579 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:46,580 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:46,580 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524606372_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:46,580 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524606372_0001_1_00_000000_0
2016-05-17 22:36:46,580 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:46,580 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:46,580 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606372_0001_1][Event:TASK_FINISHED]: vertexName=B026AAE895E44B6094B24DA6F3E436D4, taskId=task_1463524606372_0001_1_00_000000, startTime=1463524606550, finishTime=1463524606580, timeTaken=30, status=SUCCEEDED, successfulAttemptID=attempt_1463524606372_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=343932928, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524606573, Process_Duration=5, Process_End_Time=1463524606578, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:46,581 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524606372_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:46,581 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4] : 1
2016-05-17 22:36:46,581 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4]
2016-05-17 22:36:46,581 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:46,581 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524606372_0001_00_000001
2016-05-17 22:36:46,582 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606372_0001_1][Event:VERTEX_FINISHED]: vertexName=B026AAE895E44B6094B24DA6F3E436D4, vertexId=vertex_1463524606372_0001_1_00, initRequestedTime=1463524606530, initedTime=1463524606540, startRequestedTime=1463524606540, startedTime=1463524606540, finishTime=1463524606581, timeTaken=41, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=343932928, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524606573, Process_Duration=5, Process_End_Time=1463524606578, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524606550, firstTasksToStart=[ task_1463524606372_0001_1_00_000000 ], lastTaskFinishTime=1463524606579, lastTasksToFinish=[ task_1463524606372_0001_1_00_000000 ], minTaskDuration=29, maxTaskDuration=29, avgTaskDuration=29.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524606372_0001_1_00_000000 ], longestDurationTasks=[ task_1463524606372_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:46,582 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:46,582 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524606372_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:46,582 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606372_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524606372_0001_00_000001, stoppedTime=1463524606582, exitStatus=0
2016-05-17 22:36:46,582 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:46,582 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524606372_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:46,582 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:46,583 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524606372_0001_1
2016-05-17 22:36:46,583 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606372_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524606372_0001_1
2016-05-17 22:36:46,583 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 5771B51EBAB6446587C731C3BBB5E9B6 for vertex: vertex_1463524606372_0001_1_00 [B026AAE895E44B6094B24DA6F3E436D4]
2016-05-17 22:36:46,592 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524606372_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:46,592 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524606372_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:46,592 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524606372_0001_00_000001, Exiting
2016-05-17 22:36:46,592 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524606372_0001_00_000001
2016-05-17 22:36:46,592 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524606372_0001_00_000001
2016-05-17 22:36:46,594 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524606372_0001_1, queueSize=0
2016-05-17 22:36:46,594 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:46,594 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606372_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524606372_0001_1, startTime=1463524606529, finishTime=1463524606593, timeTaken=64, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=343932928, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524606573, Process_Duration=5, Process_End_Time=1463524606578, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:46,594 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524606372_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:46,595 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524606372_0001_00_000001 completed successfully
2016-05-17 22:36:46,594 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524606372_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:46,595 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524606372_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:46,595 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524606372_0001_00_000001
2016-05-17 22:36:46 Completed Dag: dag_1463524606372_0001_1
2016-05-17 22:36:46,595 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524606372_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:46,596 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524606372_0001_1_post
2016-05-17 22:36:46,596 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:46,596 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:46,596 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524606372_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:46,601 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/ALL_UNKNOWN_-1, applicationId=application_1463524606372_0001
2016-05-17 22:36:46,603 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:46,603 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:46,603 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:46,603 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:46,603 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:46,603 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:46,603 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:46,604 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:46,604 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:46,604 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:46,604 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:46,604 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:46,604 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:46,608 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/416143C490E8446DA6A7AD56617C9F20/B1931B653B6D4F4F8D941D85610A4409/.tez/application_1463524606372_0001
2016-05-17 22:36:46,608 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_-1/_temporary
2016-05-17 22:36:46,609 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.253
2016-05-17 22:36:46,674 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_UNKNOWN_-1
2016-05-17 22:36:46,693 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:46,694 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:46,718 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.002
2016-05-17 22:36:46,720 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-05-17 22:36:46,720 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:46,721 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:46,721 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_UNKNOWN_'field'
2016-05-17 22:36:46,721 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:46,721 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:46,740 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:46,741 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:46,771 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-05-17 22:36:46,775 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 22:36:46,775 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:46,775 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:46,775 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:46,787 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:46,788 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_RESULTS
2016-05-17 22:36:46,795 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:46,795 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:46,795 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_RESULTS"]
2016-05-17 22:36:46,795 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:46,796 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:46,796 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:46,797 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...tform/ALL_UNKNOWN_RESULTS
2016-05-17 22:36:46,797 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:46,808 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:46,812 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:46,812 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:46,812 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:46,822 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/AAA83B548C7943DEBC97765FF6D0D982/CF1EC9656F684F92AE3A74FB1D772351/.tez/application_1463524606812_0001 doesn't exist and is created
2016-05-17 22:36:46,837 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:46,837 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:46,838 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/AAA83B548C7943DEBC97765FF6D0D982/CF1EC9656F684F92AE3A74FB1D772351/.tez/application_1463524606812_0001
2016-05-17 22:36:46,840 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524606812_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:46,849 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:46,850 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:46,850 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:46,851 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:46,851 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/AAA83B548C7943DEBC97765FF6D0D982/CF1EC9656F684F92AE3A74FB1D772351/.tez/application_1463524606812_0001/localmode-log-dir
2016-05-17 22:36:46,851 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/AAA83B548C7943DEBC97765FF6D0D982/CF1EC9656F684F92AE3A74FB1D772351/.tez/application_1463524606812_0001/localmode-log-dir/history.txt.appattempt_1463524606812_0001_000000, maxErrors=10
2016-05-17 22:36:46,851 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:46,851 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524606812_0001_000000, appSubmitTime=1463524606840, launchTime=1463524606840
2016-05-17 22:36:46,856 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:53257
2016-05-17 22:36:46,856 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:46,856 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:46,857 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:46,858 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:46,859 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524606812_0001_000000, startTime=1463524606859
2016-05-17 22:36:46,859 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:46,860 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:46,860 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:46,938 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:46,938 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:46,938 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/ALL_UNKNOWN_RESULTS, applicationId=application_1463524606812_0001, dagName=[AAA83B548C7943DEBC97765FF6D0D982/CF1EC9656F684F92AE3A74FB1D772351] each/(1/1) ...tform/ALL_UNKNOWN_RESULTS
2016-05-17 22:36:46,953 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [AAA83B548C7943DEBC97765FF6D0D982/CF1EC9656F684F92AE3A74FB1D772351] each/(1/1) ...tform/ALL_UNKNOWN_RESULTS
2016-05-17 22:36:46,954 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524606812_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/AAA83B548C7943DEBC97765FF6D0D982/CF1EC9656F684F92AE3A74FB1D772351/.tez/application_1463524606812_0001/localmode-log-dir/dag_1463524606812_0001_1.dot
2016-05-17 22:36:46,954 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/AAA83B548C7943DEBC97765FF6D0D982/CF1EC9656F684F92AE3A74FB1D772351/.tez/application_1463524606812_0001/localmode-log-dir/dag_1463524606812_0001_1-tez-dag.pb.txt
2016-05-17 22:36:46,958 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524606812_0001_1
2016-05-17 22:36:46,958 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:46,958 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [AAA83B548C7943DEBC97765FF6D0D982/CF1EC9656F684F92AE3A74FB1D772351] each/(1/1) ...tform/ALL_UNKNOWN_RESULTS
2016-05-17 22:36:46 Running Dag: dag_1463524606812_0001_1
2016-05-17 22:36:46,969 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:46,969 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606812_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524606812_0001_1, submitTime=1463524606953
2016-05-17 22:36:46,970 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B944A5A8492441DF90877B4C507D4CD5
2016-05-17 22:36:46,970 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:46,970 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:46,970 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606812_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524606812_0001_1, initTime=1463524606970
2016-05-17 22:36:46,971 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524606812_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:46,971 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606812_0001_1][Event:DAG_STARTED]: dagID=dag_1463524606812_0001_1, startTime=1463524606971
2016-05-17 22:36:46,971 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:46,971 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524606812_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:46,971 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:46,971 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B944A5A8492441DF90877B4C507D4CD5 : {9F078D7678CC43A5B47B1B9F0B18B5E9={InputName=9F078D7678CC43A5B47B1B9F0B18B5E9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:46,971 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 9F078D7678CC43A5B47B1B9F0B18B5E9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:46,971 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5]
2016-05-17 22:36:46,972 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5]
2016-05-17 22:36:46,972 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5]
2016-05-17 22:36:46,972 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5]. Starting root input initializers: 1
2016-05-17 22:36:46,972 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/ALL_UNKNOWN_RESULTS, applicationId=application_1463524606812_0001, dagName=[AAA83B548C7943DEBC97765FF6D0D982/CF1EC9656F684F92AE3A74FB1D772351] each/(1/1) ...tform/ALL_UNKNOWN_RESULTS
2016-05-17 22:36:46,972 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:46,973 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 9F078D7678CC43A5B47B1B9F0B18B5E9 on vertex vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5]
2016-05-17 22:36:46,974 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:46,975 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 9F078D7678CC43A5B47B1B9F0B18B5E9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:46,975 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:46,975 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:46,977 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:46,977 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 9F078D7678CC43A5B47B1B9F0B18B5E9 on vertex vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5]
2016-05-17 22:36:46,977 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:46,977 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {9F078D7678CC43A5B47B1B9F0B18B5E9=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:46,977 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5] parallelism set to 1
2016-05-17 22:36:46,978 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:46,978 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606812_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524606812_0001_1_00, eventCount=1
2016-05-17 22:36:46,978 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5]
2016-05-17 22:36:46,978 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=5AB6BAAF1E744CA7A9CBB89E5FD37165, vertexId=vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:46,978 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=5AB6BAAF1E744CA7A9CBB89E5FD37165, vertexId=vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5]
2016-05-17 22:36:46,980 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=5AB6BAAF1E744CA7A9CBB89E5FD37165, vertexName=B944A5A8492441DF90877B4C507D4CD5, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:46,980 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=5AB6BAAF1E744CA7A9CBB89E5FD37165, vertexId=vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5]
2016-05-17 22:36:46,981 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:46,981 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606812_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B944A5A8492441DF90877B4C507D4CD5, vertexId=vertex_1463524606812_0001_1_00, initRequestedTime=1463524606971, initedTime=1463524606981, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:46,981 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5] with distanceFromRoot: 0
2016-05-17 22:36:46,981 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:46,982 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B944A5A8492441DF90877B4C507D4CD5
2016-05-17 22:36:46,982 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5]
2016-05-17 22:36:46,982 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:46,982 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606812_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524606812_0001_1_00, startRequestedTime=1463524606981, startedTime=1463524606981
2016-05-17 22:36:46,982 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:46,983 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:46,983 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606812_0001_1][Event:TASK_STARTED]: vertexName=B944A5A8492441DF90877B4C507D4CD5, taskId=task_1463524606812_0001_1_00_000000, scheduledTime=1463524606983, launchTime=1463524606983
2016-05-17 22:36:46,983 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524606812_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:46,983 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5]
2016-05-17 22:36:46,984 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [AAA83B548C7943DEBC97765FF6D0D982/CF1EC9656F684F92AE3A74FB1D772351] each/(1/1) ...tform/ALL_UNKNOWN_RESULTS, VertexName: B944A5A8492441DF90877B4C507D4CD5, VertexParallelism: 1, TaskAttemptID:attempt_1463524606812_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9F078D7678CC43A5B47B1B9F0B18B5E9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=5AB6BAAF1E744CA7A9CBB89E5FD37165, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:46,984 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524606812_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:46,984 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:46,985 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:46,985 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:46,985 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:46,986 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524606812_0001_00_000001
2016-05-17 22:36:46,986 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524606812_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:46,986 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524606812_0001, with dag id: dag_1463524606812_0001_1
2016-05-17 22:36:46,987 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524606812_0001_00_000001
2016-05-17 22:36:46,987 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:46,988 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524606812_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:46,988 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:46,989 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606812_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524606812_0001_00_000001, launchTime=1463524606988
2016-05-17 22:36:46,989 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524606812_0001_00_000001
2016-05-17 22:36:46,990 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524606812_0001_1_00_000000_0] to container: [container_1463524606812_0001_00_000001]
2016-05-17 22:36:46,990 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524606812_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:46,990 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524606812_0001_1_00_000000_0] started. Is using containerId: [container_1463524606812_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:46,990 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606812_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B944A5A8492441DF90877B4C507D4CD5, taskAttemptId=attempt_1463524606812_0001_1_00_000000_0, startTime=1463524606990, containerId=container_1463524606812_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524606812_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:46,991 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524606812_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:46,991 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:46,991 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524606812_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:46,991 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524606812_0001_00_000001 given task: attempt_1463524606812_0001_1_00_000000_0
2016-05-17 22:36:46,991 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524606812_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524606812_0001_1_00_000000_0
2016-05-17 22:36:46 Starting to run new task attempt: attempt_1463524606812_0001_1_00_000000_0
2016-05-17 22:36:46,991 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524606812_0001_1_00_000000_0
2016-05-17 22:36:46,991 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:46,991 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:46,991 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:46,992 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:46,992 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:46,992 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:46,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [AAA83B548C7943DEBC97765FF6D0D982/CF1EC9656F684F92AE3A74FB1D772351] each/(1/1) ...tform/ALL_UNKNOWN_RESULTS, VertexName: B944A5A8492441DF90877B4C507D4CD5, VertexParallelism: 1, TaskAttemptID:attempt_1463524606812_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9F078D7678CC43A5B47B1B9F0B18B5E9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=5AB6BAAF1E744CA7A9CBB89E5FD37165, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:46,992 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:46,992 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524606812_0001_1_00_000000_0
2016-05-17 22:36:46,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:46,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:46,993 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524606812_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:46,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=9F078D7678CC43A5B47B1B9F0B18B5E9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:46,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:46,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 9F078D7678CC43A5B47B1B9F0B18B5E9
2016-05-17 22:36:46,994 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 9F078D7678CC43A5B47B1B9F0B18B5E9
2016-05-17 22:36:46,995 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:46,995 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 9F078D7678CC43A5B47B1B9F0B18B5E9
2016-05-17 22:36:46,995 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 9F078D7678CC43A5B47B1B9F0B18B5E9
2016-05-17 22:36:46,995 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:46,996 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=5AB6BAAF1E744CA7A9CBB89E5FD37165, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:46,996 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:46,996 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 5AB6BAAF1E744CA7A9CBB89E5FD37165
2016-05-17 22:36:46,996 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:46,996 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 5AB6BAAF1E744CA7A9CBB89E5FD37165
2016-05-17 22:36:46,997 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:47,004 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B944A5A8492441DF90877B4C507D4CD5, ordinal: 0
2016-05-17 22:36:47,004 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B944A5A8492441DF90877B4C507D4CD5, mem on start (mb), free: 140, total: 328, max: 672
2016-05-17 22:36:47,005 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:47,005 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:47,005 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:47,012 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:47,012 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 5AB6BAAF1E744CA7A9CBB89E5FD37165
2016-05-17 22:36:47,012 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:47,012 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:47,012 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:47,013 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:47,013 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:47,013 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:47,014 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:47,014 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:47,015 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 9F078D7678CC43A5B47B1B9F0B18B5E9, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:47,015 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 5AB6BAAF1E744CA7A9CBB89E5FD37165, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:47,015 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:47,015 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 9F078D7678CC43A5B47B1B9F0B18B5E9
2016-05-17 22:36:47,015 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:47,015 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 9F078D7678CC43A5B47B1B9F0B18B5E9
2016-05-17 22:36:47,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 9F078D7678CC43A5B47B1B9F0B18B5E9 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:47,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:47,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:47,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:47,017 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524606812_0001_1_00_000000_0
2016-05-17 22:36:47,019 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:47,019 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@10661efb
2016-05-17 22:36:47,020 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:47,020 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 9F078D7678CC43A5B47B1B9F0B18B5E9
2016-05-17 22:36:47,020 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_RESULTS"], id: 5AB6BAAF1E744CA7A9CBB89E5FD37165
2016-05-17 22:36:47,021 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_RESULTS"]
2016-05-17 22:36:47,021 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:47,021 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:47,023 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@10661efb
2016-05-17 22:36:47,024 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:47,024 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:47,024 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B944A5A8492441DF90877B4C507D4CD5, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:47,028 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:47,028 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:47,028 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524606812_0001_1_00_000000_0
2016-05-17 22:36:47,029 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524606812_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:47,029 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524606812_0001_1_00_000000_0
2016-05-17 22:36:47,029 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B944A5A8492441DF90877B4C507D4CD5, mem on close (mb), free: 138, total: 328, max: 672
2016-05-17 22:36:47 Completed running task attempt: attempt_1463524606812_0001_1_00_000000_0
2016-05-17 22:36:47,030 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524606812_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:47,031 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=343932928, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524607024, Process_Duration=5, Process_End_Time=1463524607029, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:47,031 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606812_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B944A5A8492441DF90877B4C507D4CD5, taskAttemptId=attempt_1463524606812_0001_1_00_000000_0, startTime=1463524606990, finishTime=1463524607031, timeTaken=41, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=343932928, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524607024, Process_Duration=5, Process_End_Time=1463524607029, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:47,031 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:47,031 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:47,031 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:47,032 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524606812_0001_00_000001
2016-05-17 22:36:47,031 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524606812_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:47,032 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:47,032 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524606812_0001_1_00_000000_0
2016-05-17 22:36:47,033 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:47,033 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606812_0001_1][Event:TASK_FINISHED]: vertexName=B944A5A8492441DF90877B4C507D4CD5, taskId=task_1463524606812_0001_1_00_000000, startTime=1463524606990, finishTime=1463524607033, timeTaken=43, status=SUCCEEDED, successfulAttemptID=attempt_1463524606812_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=343932928, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524607024, Process_Duration=5, Process_End_Time=1463524607029, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:47,033 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524606812_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:47,033 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5] : 1
2016-05-17 22:36:47,033 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5]
2016-05-17 22:36:47,034 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606812_0001_1][Event:VERTEX_FINISHED]: vertexName=B944A5A8492441DF90877B4C507D4CD5, vertexId=vertex_1463524606812_0001_1_00, initRequestedTime=1463524606971, initedTime=1463524606981, startRequestedTime=1463524606981, startedTime=1463524606981, finishTime=1463524607034, timeTaken=53, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=343932928, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524607024, Process_Duration=5, Process_End_Time=1463524607029, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524606990, firstTasksToStart=[ task_1463524606812_0001_1_00_000000 ], lastTaskFinishTime=1463524607031, lastTasksToFinish=[ task_1463524606812_0001_1_00_000000 ], minTaskDuration=41, maxTaskDuration=41, avgTaskDuration=41.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524606812_0001_1_00_000000 ], longestDurationTasks=[ task_1463524606812_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:47,034 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:47,035 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524606812_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:47,035 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606812_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524606812_0001_00_000001, stoppedTime=1463524607035, exitStatus=0
2016-05-17 22:36:47,035 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524606812_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:47,035 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:47,035 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524606812_0001_1
2016-05-17 22:36:47,035 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606812_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524606812_0001_1
2016-05-17 22:36:47,035 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 5AB6BAAF1E744CA7A9CBB89E5FD37165 for vertex: vertex_1463524606812_0001_1_00 [B944A5A8492441DF90877B4C507D4CD5]
2016-05-17 22:36:47,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:47,037 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:47,042 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524606812_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:47,042 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524606812_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:47,042 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524606812_0001_00_000001, Exiting
2016-05-17 22:36:47,042 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524606812_0001_00_000001
2016-05-17 22:36:47,042 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524606812_0001_00_000001
2016-05-17 22:36:47,043 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524606812_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:47,043 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524606812_0001_00_000001 completed successfully
2016-05-17 22:36:47,046 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524606812_0001_1, queueSize=0
2016-05-17 22:36:47,046 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:47,046 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524606812_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524606812_0001_1, startTime=1463524606971, finishTime=1463524607045, timeTaken=74, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=343932928, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524607024, Process_Duration=5, Process_End_Time=1463524607029, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:47,046 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524606812_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:47,046 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524606812_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:47,046 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524606812_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:47,046 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524606812_0001_00_000001
2016-05-17 22:36:47 Completed Dag: dag_1463524606812_0001_1
2016-05-17 22:36:47,047 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524606812_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:47,047 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524606812_0001_1_post
2016-05-17 22:36:47,047 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:47,047 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:47,052 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/ALL_UNKNOWN_RESULTS, applicationId=application_1463524606812_0001
2016-05-17 22:36:47,054 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:47,054 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:47,054 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:47,054 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:47,054 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:47,055 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:47,055 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:47,055 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:47,055 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:47,055 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:47,056 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:47,056 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:47,056 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:47,060 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/AAA83B548C7943DEBC97765FF6D0D982/CF1EC9656F684F92AE3A74FB1D772351/.tez/application_1463524606812_0001
2016-05-17 22:36:47,060 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_RESULTS/_temporary
2016-05-17 22:36:47,061 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.265
2016-05-17 22:36:47,096 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:47,096 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:47,124 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_UNKNOWN_RESULTS
2016-05-17 22:36:47,146 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:47,146 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:47,179 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 22:36:47,182 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 22:36:47,182 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:47,182 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:47,182 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:47,195 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:47,196 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_ALL
2016-05-17 22:36:47,204 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:47,204 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:47,205 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_ALL"]
2016-05-17 22:36:47,205 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:47,206 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:47,207 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:47,207 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...yplatform/ALL_UNKNOWN_ALL
2016-05-17 22:36:47,207 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:47,218 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:47,222 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:47,222 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:47,222 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:47,231 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A865F52357ED4D9F913471149207E80F/E595198C115047258526790B0FBF3970/.tez/application_1463524607222_0001 doesn't exist and is created
2016-05-17 22:36:47,246 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:47,246 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:47,247 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A865F52357ED4D9F913471149207E80F/E595198C115047258526790B0FBF3970/.tez/application_1463524607222_0001
2016-05-17 22:36:47,249 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524607222_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:47,272 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:47,272 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:47,273 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:47,274 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:47,274 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A865F52357ED4D9F913471149207E80F/E595198C115047258526790B0FBF3970/.tez/application_1463524607222_0001/localmode-log-dir
2016-05-17 22:36:47,274 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A865F52357ED4D9F913471149207E80F/E595198C115047258526790B0FBF3970/.tez/application_1463524607222_0001/localmode-log-dir/history.txt.appattempt_1463524607222_0001_000000, maxErrors=10
2016-05-17 22:36:47,274 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:47,275 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524607222_0001_000000, appSubmitTime=1463524607249, launchTime=1463524607249
2016-05-17 22:36:47,279 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:47,279 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:47,280 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:51374
2016-05-17 22:36:47,280 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:47,281 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:47,283 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:47,283 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524607222_0001_000000, startTime=1463524607283
2016-05-17 22:36:47,283 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:47,283 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:47,346 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:47,346 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:47,347 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/ALL_UNKNOWN_ALL, applicationId=application_1463524607222_0001, dagName=[A865F52357ED4D9F913471149207E80F/E595198C115047258526790B0FBF3970] each/(1/1) ...yplatform/ALL_UNKNOWN_ALL
2016-05-17 22:36:47,361 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [A865F52357ED4D9F913471149207E80F/E595198C115047258526790B0FBF3970] each/(1/1) ...yplatform/ALL_UNKNOWN_ALL
2016-05-17 22:36:47,362 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524607222_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A865F52357ED4D9F913471149207E80F/E595198C115047258526790B0FBF3970/.tez/application_1463524607222_0001/localmode-log-dir/dag_1463524607222_0001_1.dot
2016-05-17 22:36:47,363 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A865F52357ED4D9F913471149207E80F/E595198C115047258526790B0FBF3970/.tez/application_1463524607222_0001/localmode-log-dir/dag_1463524607222_0001_1-tez-dag.pb.txt
2016-05-17 22:36:47,366 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524607222_0001_1
2016-05-17 22:36:47,366 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:47,366 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [A865F52357ED4D9F913471149207E80F/E595198C115047258526790B0FBF3970] each/(1/1) ...yplatform/ALL_UNKNOWN_ALL
2016-05-17 22:36:47 Running Dag: dag_1463524607222_0001_1
2016-05-17 22:36:47,378 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607222_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524607222_0001_1, submitTime=1463524607361
2016-05-17 22:36:47,378 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:47,378 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A91418D7A1224090988AB9C5934051C9
2016-05-17 22:36:47,379 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:47,379 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607222_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524607222_0001_1, initTime=1463524607378
2016-05-17 22:36:47,379 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524607222_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:47,379 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:47,380 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607222_0001_1][Event:DAG_STARTED]: dagID=dag_1463524607222_0001_1, startTime=1463524607380
2016-05-17 22:36:47,380 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:47,380 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524607222_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:47,380 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A91418D7A1224090988AB9C5934051C9 : {41F274A0FB2B4DA6B83D3B932D0D7B19={InputName=41F274A0FB2B4DA6B83D3B932D0D7B19}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:47,380 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 41F274A0FB2B4DA6B83D3B932D0D7B19, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:47,380 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9]
2016-05-17 22:36:47,380 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/ALL_UNKNOWN_ALL, applicationId=application_1463524607222_0001, dagName=[A865F52357ED4D9F913471149207E80F/E595198C115047258526790B0FBF3970] each/(1/1) ...yplatform/ALL_UNKNOWN_ALL
2016-05-17 22:36:47,381 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9]
2016-05-17 22:36:47,381 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9]
2016-05-17 22:36:47,381 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9]. Starting root input initializers: 1
2016-05-17 22:36:47,381 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:47,381 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:47,382 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 41F274A0FB2B4DA6B83D3B932D0D7B19 on vertex vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9]
2016-05-17 22:36:47,383 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:47,384 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 41F274A0FB2B4DA6B83D3B932D0D7B19 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:47,384 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:47,384 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:47,385 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:47,386 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 41F274A0FB2B4DA6B83D3B932D0D7B19 on vertex vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9]
2016-05-17 22:36:47,386 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:47,386 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {41F274A0FB2B4DA6B83D3B932D0D7B19=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:47,386 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9] parallelism set to 1
2016-05-17 22:36:47,386 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:47,386 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607222_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524607222_0001_1_00, eventCount=1
2016-05-17 22:36:47,387 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9]
2016-05-17 22:36:47,387 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D0C3C8BA96BC4F019089C8B23F3C20F3, vertexId=vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:47,387 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D0C3C8BA96BC4F019089C8B23F3C20F3, vertexId=vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9]
2016-05-17 22:36:47,389 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D0C3C8BA96BC4F019089C8B23F3C20F3, vertexName=A91418D7A1224090988AB9C5934051C9, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:47,389 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D0C3C8BA96BC4F019089C8B23F3C20F3, vertexId=vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9]
2016-05-17 22:36:47,390 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:47,390 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607222_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A91418D7A1224090988AB9C5934051C9, vertexId=vertex_1463524607222_0001_1_00, initRequestedTime=1463524607380, initedTime=1463524607390, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:47,390 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9] with distanceFromRoot: 0
2016-05-17 22:36:47,390 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:47,390 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A91418D7A1224090988AB9C5934051C9
2016-05-17 22:36:47,390 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9]
2016-05-17 22:36:47,391 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:47,391 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607222_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524607222_0001_1_00, startRequestedTime=1463524607390, startedTime=1463524607390
2016-05-17 22:36:47,391 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:47,392 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:47,392 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607222_0001_1][Event:TASK_STARTED]: vertexName=A91418D7A1224090988AB9C5934051C9, taskId=task_1463524607222_0001_1_00_000000, scheduledTime=1463524607391, launchTime=1463524607391
2016-05-17 22:36:47,392 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524607222_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:47,392 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9]
2016-05-17 22:36:47,392 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A865F52357ED4D9F913471149207E80F/E595198C115047258526790B0FBF3970] each/(1/1) ...yplatform/ALL_UNKNOWN_ALL, VertexName: A91418D7A1224090988AB9C5934051C9, VertexParallelism: 1, TaskAttemptID:attempt_1463524607222_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=41F274A0FB2B4DA6B83D3B932D0D7B19, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D0C3C8BA96BC4F019089C8B23F3C20F3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:47,393 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524607222_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:47,393 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:47,393 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:47,394 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:47,394 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:47,394 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524607222_0001_00_000001
2016-05-17 22:36:47,394 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524607222_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:47,396 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524607222_0001_00_000001
2016-05-17 22:36:47,396 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:47,396 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524607222_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:47,396 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:47,397 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607222_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524607222_0001_00_000001, launchTime=1463524607396
2016-05-17 22:36:47,397 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524607222_0001_00_000001
2016-05-17 22:36:47,397 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524607222_0001_1_00_000000_0] to container: [container_1463524607222_0001_00_000001]
2016-05-17 22:36:47,397 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524607222_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:47,398 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524607222_0001_1_00_000000_0] started. Is using containerId: [container_1463524607222_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:47,398 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607222_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A91418D7A1224090988AB9C5934051C9, taskAttemptId=attempt_1463524607222_0001_1_00_000000_0, startTime=1463524607398, containerId=container_1463524607222_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524607222_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:47,398 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524607222_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:47,398 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524607222_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:47,398 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:47,399 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524607222_0001_00_000001 given task: attempt_1463524607222_0001_1_00_000000_0
2016-05-17 22:36:47,399 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524607222_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524607222_0001_1_00_000000_0
2016-05-17 22:36:47,399 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524607222_0001, with dag id: dag_1463524607222_0001_1
2016-05-17 22:36:47 Starting to run new task attempt: attempt_1463524607222_0001_1_00_000000_0
2016-05-17 22:36:47,399 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524607222_0001_1_00_000000_0
2016-05-17 22:36:47,399 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:47,400 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:47,400 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:47,400 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:47,400 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:47,400 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:47,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A865F52357ED4D9F913471149207E80F/E595198C115047258526790B0FBF3970] each/(1/1) ...yplatform/ALL_UNKNOWN_ALL, VertexName: A91418D7A1224090988AB9C5934051C9, VertexParallelism: 1, TaskAttemptID:attempt_1463524607222_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=41F274A0FB2B4DA6B83D3B932D0D7B19, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D0C3C8BA96BC4F019089C8B23F3C20F3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:47,400 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:47,401 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524607222_0001_1_00_000000_0
2016-05-17 22:36:47,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:47,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:47,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=41F274A0FB2B4DA6B83D3B932D0D7B19, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:47,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:47,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 41F274A0FB2B4DA6B83D3B932D0D7B19
2016-05-17 22:36:47,402 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 41F274A0FB2B4DA6B83D3B932D0D7B19
2016-05-17 22:36:47,402 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524607222_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:47,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:47,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D0C3C8BA96BC4F019089C8B23F3C20F3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:47,404 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:47,404 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:47,405 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 41F274A0FB2B4DA6B83D3B932D0D7B19
2016-05-17 22:36:47,405 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 41F274A0FB2B4DA6B83D3B932D0D7B19
2016-05-17 22:36:47,405 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:47,404 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D0C3C8BA96BC4F019089C8B23F3C20F3
2016-05-17 22:36:47,405 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:47,405 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D0C3C8BA96BC4F019089C8B23F3C20F3
2016-05-17 22:36:47,410 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A91418D7A1224090988AB9C5934051C9, ordinal: 0
2016-05-17 22:36:47,410 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A91418D7A1224090988AB9C5934051C9, mem on start (mb), free: 223, total: 325, max: 672
2016-05-17 22:36:47,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:47,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:47,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:47,415 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:47,415 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D0C3C8BA96BC4F019089C8B23F3C20F3
2016-05-17 22:36:47,415 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:47,415 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:47,415 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:47,415 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:47,415 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:47,415 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:47,416 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:47,416 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:47,416 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 41F274A0FB2B4DA6B83D3B932D0D7B19, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:47,416 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D0C3C8BA96BC4F019089C8B23F3C20F3, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:47,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:47,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 41F274A0FB2B4DA6B83D3B932D0D7B19 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:47,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 41F274A0FB2B4DA6B83D3B932D0D7B19
2016-05-17 22:36:47,417 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:47,417 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 41F274A0FB2B4DA6B83D3B932D0D7B19
2016-05-17 22:36:47,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:47,417 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:47,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:47,418 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524607222_0001_1_00_000000_0
2016-05-17 22:36:47,419 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:47,420 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5690699
2016-05-17 22:36:47,420 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:47,421 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5690699
2016-05-17 22:36:47,422 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:47,422 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:47,422 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 41F274A0FB2B4DA6B83D3B932D0D7B19
2016-05-17 22:36:47,422 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_ALL"], id: D0C3C8BA96BC4F019089C8B23F3C20F3
2016-05-17 22:36:47,422 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_ALL"]
2016-05-17 22:36:47,423 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:47,423 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:47,423 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A91418D7A1224090988AB9C5934051C9, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:47,425 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:47,425 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:47,426 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524607222_0001_1_00_000000_0
2016-05-17 22:36:47,426 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524607222_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:47,426 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524607222_0001_1_00_000000_0
2016-05-17 22:36:47,427 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A91418D7A1224090988AB9C5934051C9, mem on close (mb), free: 221, total: 325, max: 672
2016-05-17 22:36:47 Completed running task attempt: attempt_1463524607222_0001_1_00_000000_0
2016-05-17 22:36:47,427 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524607222_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:47,428 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=340787200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524607423, Process_Duration=3, Process_End_Time=1463524607426, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:47,428 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607222_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A91418D7A1224090988AB9C5934051C9, taskAttemptId=attempt_1463524607222_0001_1_00_000000_0, startTime=1463524607398, finishTime=1463524607428, timeTaken=30, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=340787200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524607423, Process_Duration=3, Process_End_Time=1463524607426, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:47,429 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:47,429 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524607222_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:47,428 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:47,429 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:47,429 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:47,429 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524607222_0001_00_000001
2016-05-17 22:36:47,429 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524607222_0001_1_00_000000_0
2016-05-17 22:36:47,430 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:47,430 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607222_0001_1][Event:TASK_FINISHED]: vertexName=A91418D7A1224090988AB9C5934051C9, taskId=task_1463524607222_0001_1_00_000000, startTime=1463524607398, finishTime=1463524607430, timeTaken=32, status=SUCCEEDED, successfulAttemptID=attempt_1463524607222_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=340787200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524607423, Process_Duration=3, Process_End_Time=1463524607426, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:47,430 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524607222_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:47,431 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9] : 1
2016-05-17 22:36:47,431 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9]
2016-05-17 22:36:47,431 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:47,432 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607222_0001_1][Event:VERTEX_FINISHED]: vertexName=A91418D7A1224090988AB9C5934051C9, vertexId=vertex_1463524607222_0001_1_00, initRequestedTime=1463524607380, initedTime=1463524607390, startRequestedTime=1463524607390, startedTime=1463524607390, finishTime=1463524607431, timeTaken=41, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=340787200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524607423, Process_Duration=3, Process_End_Time=1463524607426, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524607398, firstTasksToStart=[ task_1463524607222_0001_1_00_000000 ], lastTaskFinishTime=1463524607428, lastTasksToFinish=[ task_1463524607222_0001_1_00_000000 ], minTaskDuration=30, maxTaskDuration=30, avgTaskDuration=30.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524607222_0001_1_00_000000 ], longestDurationTasks=[ task_1463524607222_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:47,432 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:47,432 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524607222_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:47,432 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607222_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524607222_0001_00_000001, stoppedTime=1463524607432, exitStatus=0
2016-05-17 22:36:47,432 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:47,432 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524607222_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:47,432 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:47,432 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524607222_0001_1
2016-05-17 22:36:47,432 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607222_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524607222_0001_1
2016-05-17 22:36:47,433 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: D0C3C8BA96BC4F019089C8B23F3C20F3 for vertex: vertex_1463524607222_0001_1_00 [A91418D7A1224090988AB9C5934051C9]
2016-05-17 22:36:47,440 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524607222_0001_1, queueSize=0
2016-05-17 22:36:47,440 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:47,440 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607222_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524607222_0001_1, startTime=1463524607380, finishTime=1463524607439, timeTaken=59, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=340787200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524607423, Process_Duration=3, Process_End_Time=1463524607426, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:47,440 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524607222_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:47,440 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524607222_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:47,440 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524607222_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:47,441 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524607222_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:47,440 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524607222_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:36:47,441 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524607222_0001_00_000001, Exiting
2016-05-17 22:36:47,441 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524607222_0001_00_000001
2016-05-17 22:36:47,441 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524607222_0001_00_000001
2016-05-17 22:36:47 Completed Dag: dag_1463524607222_0001_1
2016-05-17 22:36:47,441 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524607222_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:47,441 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524607222_0001_1_post
2016-05-17 22:36:47,441 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:47,441 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:47,441 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524607222_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:36:47,442 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524607222_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:47,442 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524607222_0001_00_000001 completed successfully
2016-05-17 22:36:47,442 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524607222_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:36:47,444 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:47,444 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:47,448 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/ALL_UNKNOWN_ALL, applicationId=application_1463524607222_0001
2016-05-17 22:36:47,450 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:47,450 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:47,450 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:47,450 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:47,450 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:47,451 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:47,451 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:47,452 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:47,452 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:47,452 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:47,452 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:47,452 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:47,453 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:47,456 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A865F52357ED4D9F913471149207E80F/E595198C115047258526790B0FBF3970/.tez/application_1463524607222_0001
2016-05-17 22:36:47,456 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_ALL/_temporary
2016-05-17 22:36:47,456 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.252
2016-05-17 22:36:47,531 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_UNKNOWN_ALL
2016-05-17 22:36:47,551 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:47,551 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:47,581 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-17 22:36:47,584 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 22:36:47,584 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:47,585 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:47,585 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:47,598 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:47,598 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_REPLACE
2016-05-17 22:36:47,606 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:47,606 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:47,606 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_REPLACE"]
2016-05-17 22:36:47,606 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:47,606 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:47,606 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:47,607 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-05-17 22:36:47,607 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:47,617 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:47,622 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:47,622 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:47,622 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:47,633 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/10C012B5DCD54EDB8AD12A19F15D6E46/C8B05741415D430890C73A095399C478/.tez/application_1463524607622_0001 doesn't exist and is created
2016-05-17 22:36:47,648 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:47,648 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:47,648 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/10C012B5DCD54EDB8AD12A19F15D6E46/C8B05741415D430890C73A095399C478/.tez/application_1463524607622_0001
2016-05-17 22:36:47,651 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524607622_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:47,660 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:47,661 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:47,661 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:47,662 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:47,662 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/10C012B5DCD54EDB8AD12A19F15D6E46/C8B05741415D430890C73A095399C478/.tez/application_1463524607622_0001/localmode-log-dir
2016-05-17 22:36:47,662 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/10C012B5DCD54EDB8AD12A19F15D6E46/C8B05741415D430890C73A095399C478/.tez/application_1463524607622_0001/localmode-log-dir/history.txt.appattempt_1463524607622_0001_000000, maxErrors=10
2016-05-17 22:36:47,662 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:47,663 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524607622_0001_000000, appSubmitTime=1463524607650, launchTime=1463524607651
2016-05-17 22:36:47,666 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:47,666 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:47,667 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:56248
2016-05-17 22:36:47,667 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:47,669 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:47,669 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:47,671 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524607622_0001_000000, startTime=1463524607671
2016-05-17 22:36:47,671 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:47,671 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:47,748 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:47,748 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:47,748 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/ALL_UNKNOWN_REPLACE, applicationId=application_1463524607622_0001, dagName=[10C012B5DCD54EDB8AD12A19F15D6E46/C8B05741415D430890C73A095399C478] each/(1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-05-17 22:36:47,762 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [10C012B5DCD54EDB8AD12A19F15D6E46/C8B05741415D430890C73A095399C478] each/(1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-05-17 22:36:47,763 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524607622_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/10C012B5DCD54EDB8AD12A19F15D6E46/C8B05741415D430890C73A095399C478/.tez/application_1463524607622_0001/localmode-log-dir/dag_1463524607622_0001_1.dot
2016-05-17 22:36:47,764 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/10C012B5DCD54EDB8AD12A19F15D6E46/C8B05741415D430890C73A095399C478/.tez/application_1463524607622_0001/localmode-log-dir/dag_1463524607622_0001_1-tez-dag.pb.txt
2016-05-17 22:36:47,768 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524607622_0001_1
2016-05-17 22:36:47,768 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:47,768 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [10C012B5DCD54EDB8AD12A19F15D6E46/C8B05741415D430890C73A095399C478] each/(1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-05-17 22:36:47 Running Dag: dag_1463524607622_0001_1
2016-05-17 22:36:47,780 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:47,780 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607622_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524607622_0001_1, submitTime=1463524607762
2016-05-17 22:36:47,781 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A8223B6441C54E7C974FCC45261088E7
2016-05-17 22:36:47,781 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:47,781 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:47,781 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607622_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524607622_0001_1, initTime=1463524607780
2016-05-17 22:36:47,781 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524607622_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:47,781 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607622_0001_1][Event:DAG_STARTED]: dagID=dag_1463524607622_0001_1, startTime=1463524607781
2016-05-17 22:36:47,782 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:47,782 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:47,782 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524607622_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:47,782 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/ALL_UNKNOWN_REPLACE, applicationId=application_1463524607622_0001, dagName=[10C012B5DCD54EDB8AD12A19F15D6E46/C8B05741415D430890C73A095399C478] each/(1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-05-17 22:36:47,782 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A8223B6441C54E7C974FCC45261088E7 : {A2D946258B314E8B8C2CE636E1C2E6AB={InputName=A2D946258B314E8B8C2CE636E1C2E6AB}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:47,782 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A2D946258B314E8B8C2CE636E1C2E6AB, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:47,782 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7]
2016-05-17 22:36:47,782 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7]
2016-05-17 22:36:47,782 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7]
2016-05-17 22:36:47,782 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7]. Starting root input initializers: 1
2016-05-17 22:36:47,783 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:47,783 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A2D946258B314E8B8C2CE636E1C2E6AB on vertex vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7]
2016-05-17 22:36:47,784 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:47,785 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A2D946258B314E8B8C2CE636E1C2E6AB asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:47,785 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:47,785 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:47,786 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:47,787 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A2D946258B314E8B8C2CE636E1C2E6AB on vertex vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7]
2016-05-17 22:36:47,787 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:47,787 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A2D946258B314E8B8C2CE636E1C2E6AB=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:47,788 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7] parallelism set to 1
2016-05-17 22:36:47,788 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:47,788 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607622_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524607622_0001_1_00, eventCount=1
2016-05-17 22:36:47,788 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7]
2016-05-17 22:36:47,788 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=BCCC5CF3B28D4E14B592E4F15F455743, vertexId=vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:47,788 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=BCCC5CF3B28D4E14B592E4F15F455743, vertexId=vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7]
2016-05-17 22:36:47,790 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=BCCC5CF3B28D4E14B592E4F15F455743, vertexName=A8223B6441C54E7C974FCC45261088E7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:47,790 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=BCCC5CF3B28D4E14B592E4F15F455743, vertexId=vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7]
2016-05-17 22:36:47,791 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607622_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A8223B6441C54E7C974FCC45261088E7, vertexId=vertex_1463524607622_0001_1_00, initRequestedTime=1463524607782, initedTime=1463524607791, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:47,791 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7] with distanceFromRoot: 0
2016-05-17 22:36:47,791 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:47,791 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:47,791 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A8223B6441C54E7C974FCC45261088E7
2016-05-17 22:36:47,792 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7]
2016-05-17 22:36:47,792 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607622_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524607622_0001_1_00, startRequestedTime=1463524607791, startedTime=1463524607791
2016-05-17 22:36:47,792 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:47,792 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:47,792 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:47,792 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607622_0001_1][Event:TASK_STARTED]: vertexName=A8223B6441C54E7C974FCC45261088E7, taskId=task_1463524607622_0001_1_00_000000, scheduledTime=1463524607792, launchTime=1463524607792
2016-05-17 22:36:47,792 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524607622_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:47,792 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7]
2016-05-17 22:36:47,793 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [10C012B5DCD54EDB8AD12A19F15D6E46/C8B05741415D430890C73A095399C478] each/(1/1) ...tform/ALL_UNKNOWN_REPLACE, VertexName: A8223B6441C54E7C974FCC45261088E7, VertexParallelism: 1, TaskAttemptID:attempt_1463524607622_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A2D946258B314E8B8C2CE636E1C2E6AB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BCCC5CF3B28D4E14B592E4F15F455743, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:47,793 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524607622_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:47,793 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:47,793 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:47,794 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:47,794 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:47,794 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524607622_0001_00_000001
2016-05-17 22:36:47,794 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524607622_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:47,796 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524607622_0001_00_000001
2016-05-17 22:36:47,796 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:47,796 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524607622_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:47,797 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:47,797 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607622_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524607622_0001_00_000001, launchTime=1463524607796
2016-05-17 22:36:47,797 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524607622_0001_00_000001
2016-05-17 22:36:47,798 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524607622_0001_1_00_000000_0] to container: [container_1463524607622_0001_00_000001]
2016-05-17 22:36:47,798 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524607622_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:47,798 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524607622_0001_00_000001 given task: attempt_1463524607622_0001_1_00_000000_0
2016-05-17 22:36:47,798 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524607622_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524607622_0001_1_00_000000_0
2016-05-17 22:36:47,798 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524607622_0001_1_00_000000_0] started. Is using containerId: [container_1463524607622_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:47,799 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:47,799 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607622_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A8223B6441C54E7C974FCC45261088E7, taskAttemptId=attempt_1463524607622_0001_1_00_000000_0, startTime=1463524607798, containerId=container_1463524607622_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524607622_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:47,799 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524607622_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:47,799 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524607622_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:47 Starting to run new task attempt: attempt_1463524607622_0001_1_00_000000_0
2016-05-17 22:36:47,799 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524607622_0001_1_00_000000_0
2016-05-17 22:36:47,800 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:47,800 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:47,800 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:47,800 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:47,800 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:47,800 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524607622_0001, with dag id: dag_1463524607622_0001_1
2016-05-17 22:36:47,800 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:47,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [10C012B5DCD54EDB8AD12A19F15D6E46/C8B05741415D430890C73A095399C478] each/(1/1) ...tform/ALL_UNKNOWN_REPLACE, VertexName: A8223B6441C54E7C974FCC45261088E7, VertexParallelism: 1, TaskAttemptID:attempt_1463524607622_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A2D946258B314E8B8C2CE636E1C2E6AB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BCCC5CF3B28D4E14B592E4F15F455743, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:47,801 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:47,801 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524607622_0001_1_00_000000_0
2016-05-17 22:36:47,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:47,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:47,802 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524607622_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:47,802 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A2D946258B314E8B8C2CE636E1C2E6AB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:47,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:47,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A2D946258B314E8B8C2CE636E1C2E6AB
2016-05-17 22:36:47,805 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A2D946258B314E8B8C2CE636E1C2E6AB
2016-05-17 22:36:47,803 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BCCC5CF3B28D4E14B592E4F15F455743, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:47,802 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:47,808 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:47,809 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:47,810 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:47,810 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:47,810 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BCCC5CF3B28D4E14B592E4F15F455743
2016-05-17 22:36:47,810 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:47,810 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: BCCC5CF3B28D4E14B592E4F15F455743
2016-05-17 22:36:47,812 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:47,812 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A2D946258B314E8B8C2CE636E1C2E6AB
2016-05-17 22:36:47,812 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A2D946258B314E8B8C2CE636E1C2E6AB
2016-05-17 22:36:47,813 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A8223B6441C54E7C974FCC45261088E7, ordinal: 0
2016-05-17 22:36:47,814 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A8223B6441C54E7C974FCC45261088E7, mem on start (mb), free: 144, total: 325, max: 672
2016-05-17 22:36:47,814 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:47,814 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:47,814 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:47,820 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:47,820 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BCCC5CF3B28D4E14B592E4F15F455743
2016-05-17 22:36:47,820 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:47,820 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:47,820 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:47,820 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:47,820 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:47,820 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:47,821 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:47,821 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:47,821 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A2D946258B314E8B8C2CE636E1C2E6AB, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:47,821 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BCCC5CF3B28D4E14B592E4F15F455743, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:47,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:47,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A2D946258B314E8B8C2CE636E1C2E6AB being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:47,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A2D946258B314E8B8C2CE636E1C2E6AB
2016-05-17 22:36:47,821 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:47,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A2D946258B314E8B8C2CE636E1C2E6AB
2016-05-17 22:36:47,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:47,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:47,822 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:47,822 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524607622_0001_1_00_000000_0
2016-05-17 22:36:47,822 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:47,822 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3970bf2
2016-05-17 22:36:47,823 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3970bf2
2016-05-17 22:36:47,823 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:47,823 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:47,825 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:47,825 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: A2D946258B314E8B8C2CE636E1C2E6AB
2016-05-17 22:36:47,825 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_REPLACE"], id: BCCC5CF3B28D4E14B592E4F15F455743
2016-05-17 22:36:47,825 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_REPLACE"]
2016-05-17 22:36:47,826 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:47,826 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:47,826 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A8223B6441C54E7C974FCC45261088E7, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:47,826 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:47,827 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:47,827 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:47,827 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:47,828 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524607622_0001_1_00_000000_0
2016-05-17 22:36:47,828 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524607622_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:47,829 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524607622_0001_1_00_000000_0
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:47,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=340787200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524607826, Process_Duration=3, Process_End_Time=1463524607829, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:36:47,830 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:47,831 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607622_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A8223B6441C54E7C974FCC45261088E7, taskAttemptId=attempt_1463524607622_0001_1_00_000000_0, startTime=1463524607798, finishTime=1463524607830, timeTaken=32, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=340787200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524607826, Process_Duration=3, Process_End_Time=1463524607829, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:36:47,831 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524607622_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:36:47,831 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524607622_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:36:47,831 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:47,831 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607622_0001_1][Event:TASK_FINISHED]: vertexName=A8223B6441C54E7C974FCC45261088E7, taskId=task_1463524607622_0001_1_00_000000, startTime=1463524607798, finishTime=1463524607831, timeTaken=33, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
], counters=Counters: 0
2016-05-17 22:36:47,831 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524607622_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:36:47,832 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7] : 1
2016-05-17 22:36:47,832 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7] because task failed: task_1463524607622_0001_1_00_000000
2016-05-17 22:36:47,832 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:36:47,832 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:47,832 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:36:47,832 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7]
2016-05-17 22:36:47,831 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A8223B6441C54E7C974FCC45261088E7, mem on close (mb), free: 141, total: 325, max: 672
2016-05-17 22:36:47,832 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=BCCC5CF3B28D4E14B592E4F15F455743, vertexId=vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7]
2016-05-17 22:36:47,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=A8223B6441C54E7C974FCC45261088E7, index=0
2016-05-17 22:36:47,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=A8223B6441C54E7C974FCC45261088E7, sourceVertex=A2D946258B314E8B8C2CE636E1C2E6AB
2016-05-17 22:36:47,833 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=A8223B6441C54E7C974FCC45261088E7, sourceVertex=BCCC5CF3B28D4E14B592E4F15F455743
2016-05-17 22:36:47,833 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524607622_0001_00_000001
2016-05-17 22:36:47,833 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524607622_0001_00_000001
2016-05-17 22:36:47,833 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607622_0001_1][Event:VERTEX_FINISHED]: vertexName=A8223B6441C54E7C974FCC45261088E7, vertexId=vertex_1463524607622_0001_1_00, initRequestedTime=1463524607782, initedTime=1463524607791, startRequestedTime=1463524607791, startedTime=1463524607791, finishTime=1463524607833, timeTaken=42, status=FAILED, diagnostics=Task failed, taskId=task_1463524607622_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524607798, firstTasksToStart=[ task_1463524607622_0001_1_00_000000 ], lastTaskFinishTime=1463524607830, lastTasksToFinish=[ task_1463524607622_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:36:47,833 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:36:47,833 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607622_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524607622_0001_00_000001, stoppedTime=1463524607833, exitStatus=0
2016-05-17 22:36:47,834 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524607622_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:47,834 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524607622_0001_1_00_000000_0 failed: true container: container_1463524607622_0001_00_000001 numFailedTAs: 0
2016-05-17 22:36:47,834 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:36:47,834 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:36:47,834 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524607622_0001_1
2016-05-17 22:36:47,834 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: BCCC5CF3B28D4E14B592E4F15F455743 for vertex: vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7]
2016-05-17 22:36:47,834 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524607622_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:36:47,834 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524607622_0001_00_000001 completed but with errors
2016-05-17 22:36:47,834 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:47,835 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524607622_0001_1, queueSize=0
2016-05-17 22:36:47,835 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607622_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524607622_0001_1, startTime=1463524607781, finishTime=1463524607834, timeTaken=53, status=FAILED, diagnostics=Vertex failed, vertexName=A8223B6441C54E7C974FCC45261088E7, vertexId=vertex_1463524607622_0001_1_00, diagnostics=[Task failed, taskId=task_1463524607622_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524607622_0001_1_00 [A8223B6441C54E7C974FCC45261088E7] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:36:47,835 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:47,835 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524607622_0001_1 finished with state: FAILED
2016-05-17 22:36:47,835 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524607622_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:47,835 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:47,835 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524607622_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:36:47,835 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524607622_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:47 Completed Dag: dag_1463524607622_0001_1
2016-05-17 22:36:47,836 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524607622_0001_00_000001
2016-05-17 22:36:47,836 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524607622_0001_1, dagState=FAILED
2016-05-17 22:36:47,837 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524607622_0001_1_post
2016-05-17 22:36:47,837 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:47,837 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:47,838 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524607622_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:36:47,844 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/ALL_UNKNOWN_REPLACE, applicationId=application_1463524607622_0001
2016-05-17 22:36:47,846 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:47,846 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:47,846 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:47,846 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:47,846 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:47,846 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:47,847 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:47,847 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:47,847 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:47,847 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:47,847 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:47,847 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:47,847 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:47,849 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/10C012B5DCD54EDB8AD12A19F15D6E46/C8B05741415D430890C73A095399C478/.tez/application_1463524607622_0001
2016-05-17 22:36:47,850 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping all jobs
2016-05-17 22:36:47,850 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping: (1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-05-17 22:36:47,851 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopped all jobs
2016-05-17 22:36:47,851 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_REPLACE/_temporary
2016-05-17 22:36:47,851 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.245
2016-05-17 22:36:47,851 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_UNKNOWN_REPLACE
2016-05-17 22:36:47,882 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:47,882 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:47,921 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 22:36:47,928 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 22:36:47,928 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:47,928 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:47,928 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:47,941 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:47,942 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_SWAP
2016-05-17 22:36:47,950 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:47,950 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:47,950 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_SWAP"]
2016-05-17 22:36:47,950 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:47,950 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:47,950 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:47,951 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...platform/ALL_UNKNOWN_SWAP
2016-05-17 22:36:47,951 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:47,962 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:47,967 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:47,967 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:47,967 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:47,976 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B924C2093D174AF6BB9897202111CF88/D095F91091834CA3A0F0EDAF77477367/.tez/application_1463524607967_0001 doesn't exist and is created
2016-05-17 22:36:47,992 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:47,992 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:47,992 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B924C2093D174AF6BB9897202111CF88/D095F91091834CA3A0F0EDAF77477367/.tez/application_1463524607967_0001
2016-05-17 22:36:47,994 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524607967_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:48,003 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:48,003 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:48,004 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:48,004 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:48,004 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B924C2093D174AF6BB9897202111CF88/D095F91091834CA3A0F0EDAF77477367/.tez/application_1463524607967_0001/localmode-log-dir
2016-05-17 22:36:48,005 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B924C2093D174AF6BB9897202111CF88/D095F91091834CA3A0F0EDAF77477367/.tez/application_1463524607967_0001/localmode-log-dir/history.txt.appattempt_1463524607967_0001_000000, maxErrors=10
2016-05-17 22:36:48,005 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:48,005 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524607967_0001_000000, appSubmitTime=1463524607994, launchTime=1463524607994
2016-05-17 22:36:48,008 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:48,009 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:48,010 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:48,010 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:58200
2016-05-17 22:36:48,011 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:48,012 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:48,013 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524607967_0001_000000, startTime=1463524608013
2016-05-17 22:36:48,014 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:48,014 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:48,092 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:48,092 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:48,092 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/ALL_UNKNOWN_SWAP, applicationId=application_1463524607967_0001, dagName=[B924C2093D174AF6BB9897202111CF88/D095F91091834CA3A0F0EDAF77477367] each/(1/1) ...platform/ALL_UNKNOWN_SWAP
2016-05-17 22:36:48,107 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [B924C2093D174AF6BB9897202111CF88/D095F91091834CA3A0F0EDAF77477367] each/(1/1) ...platform/ALL_UNKNOWN_SWAP
2016-05-17 22:36:48,108 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524607967_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B924C2093D174AF6BB9897202111CF88/D095F91091834CA3A0F0EDAF77477367/.tez/application_1463524607967_0001/localmode-log-dir/dag_1463524607967_0001_1.dot
2016-05-17 22:36:48,108 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B924C2093D174AF6BB9897202111CF88/D095F91091834CA3A0F0EDAF77477367/.tez/application_1463524607967_0001/localmode-log-dir/dag_1463524607967_0001_1-tez-dag.pb.txt
2016-05-17 22:36:48,112 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524607967_0001_1
2016-05-17 22:36:48,112 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:48,112 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [B924C2093D174AF6BB9897202111CF88/D095F91091834CA3A0F0EDAF77477367] each/(1/1) ...platform/ALL_UNKNOWN_SWAP
2016-05-17 22:36:48 Running Dag: dag_1463524607967_0001_1
2016-05-17 22:36:48,123 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:48,123 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607967_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524607967_0001_1, submitTime=1463524608107
2016-05-17 22:36:48,124 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E8652D2A4D9B4B248887E2304DBD2B4F
2016-05-17 22:36:48,124 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:48,124 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:48,125 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607967_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524607967_0001_1, initTime=1463524608124
2016-05-17 22:36:48,125 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524607967_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:48,125 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607967_0001_1][Event:DAG_STARTED]: dagID=dag_1463524607967_0001_1, startTime=1463524608125
2016-05-17 22:36:48,125 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:48,125 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524607967_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:48,125 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E8652D2A4D9B4B248887E2304DBD2B4F : {BBB86E7BE2DD498A89EBD8DC0258236D={InputName=BBB86E7BE2DD498A89EBD8DC0258236D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:48,125 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: BBB86E7BE2DD498A89EBD8DC0258236D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:48,125 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F]
2016-05-17 22:36:48,125 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F]
2016-05-17 22:36:48,125 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F]
2016-05-17 22:36:48,125 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F]. Starting root input initializers: 1
2016-05-17 22:36:48,126 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:48,126 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:48,126 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: BBB86E7BE2DD498A89EBD8DC0258236D on vertex vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F]
2016-05-17 22:36:48,126 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/ALL_UNKNOWN_SWAP, applicationId=application_1463524607967_0001, dagName=[B924C2093D174AF6BB9897202111CF88/D095F91091834CA3A0F0EDAF77477367] each/(1/1) ...platform/ALL_UNKNOWN_SWAP
2016-05-17 22:36:48,128 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:48,128 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input BBB86E7BE2DD498A89EBD8DC0258236D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:48,140 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:48,140 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:48,141 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:48,142 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: BBB86E7BE2DD498A89EBD8DC0258236D on vertex vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F]
2016-05-17 22:36:48,142 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:48,142 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {BBB86E7BE2DD498A89EBD8DC0258236D=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:48,142 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F] parallelism set to 1
2016-05-17 22:36:48,142 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:48,142 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607967_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524607967_0001_1_00, eventCount=1
2016-05-17 22:36:48,143 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F]
2016-05-17 22:36:48,143 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=35A08E7CCDE24215A3CB0C6E9FCB37BF, vertexId=vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:48,143 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=35A08E7CCDE24215A3CB0C6E9FCB37BF, vertexId=vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F]
2016-05-17 22:36:48,145 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=35A08E7CCDE24215A3CB0C6E9FCB37BF, vertexName=E8652D2A4D9B4B248887E2304DBD2B4F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:48,145 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=35A08E7CCDE24215A3CB0C6E9FCB37BF, vertexId=vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F]
2016-05-17 22:36:48,146 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:48,146 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607967_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E8652D2A4D9B4B248887E2304DBD2B4F, vertexId=vertex_1463524607967_0001_1_00, initRequestedTime=1463524608125, initedTime=1463524608146, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:48,146 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F] with distanceFromRoot: 0
2016-05-17 22:36:48,146 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:48,147 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E8652D2A4D9B4B248887E2304DBD2B4F
2016-05-17 22:36:48,147 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F]
2016-05-17 22:36:48,147 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:48,147 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607967_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524607967_0001_1_00, startRequestedTime=1463524608146, startedTime=1463524608146
2016-05-17 22:36:48,148 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:48,148 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:48,148 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607967_0001_1][Event:TASK_STARTED]: vertexName=E8652D2A4D9B4B248887E2304DBD2B4F, taskId=task_1463524607967_0001_1_00_000000, scheduledTime=1463524608148, launchTime=1463524608148
2016-05-17 22:36:48,148 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524607967_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:48,149 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F]
2016-05-17 22:36:48,149 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B924C2093D174AF6BB9897202111CF88/D095F91091834CA3A0F0EDAF77477367] each/(1/1) ...platform/ALL_UNKNOWN_SWAP, VertexName: E8652D2A4D9B4B248887E2304DBD2B4F, VertexParallelism: 1, TaskAttemptID:attempt_1463524607967_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BBB86E7BE2DD498A89EBD8DC0258236D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=35A08E7CCDE24215A3CB0C6E9FCB37BF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:48,149 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524607967_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:48,149 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:48,150 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:48,150 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:48,151 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:48,151 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524607967_0001_00_000001
2016-05-17 22:36:48,151 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524607967_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:48,153 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524607967_0001_00_000001
2016-05-17 22:36:48,153 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:48,154 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524607967_0001, with dag id: dag_1463524607967_0001_1
2016-05-17 22:36:48,154 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524607967_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:48,154 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:48,154 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607967_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524607967_0001_00_000001, launchTime=1463524608154
2016-05-17 22:36:48,155 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524607967_0001_00_000001
2016-05-17 22:36:48,156 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524607967_0001_1_00_000000_0] to container: [container_1463524607967_0001_00_000001]
2016-05-17 22:36:48,156 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524607967_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:48,156 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524607967_0001_1_00_000000_0] started. Is using containerId: [container_1463524607967_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:48,156 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607967_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E8652D2A4D9B4B248887E2304DBD2B4F, taskAttemptId=attempt_1463524607967_0001_1_00_000000_0, startTime=1463524608156, containerId=container_1463524607967_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524607967_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:48,157 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524607967_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:48,156 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:48,157 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524607967_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:48,157 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524607967_0001_00_000001 given task: attempt_1463524607967_0001_1_00_000000_0
2016-05-17 22:36:48,157 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524607967_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524607967_0001_1_00_000000_0
2016-05-17 22:36:48 Starting to run new task attempt: attempt_1463524607967_0001_1_00_000000_0
2016-05-17 22:36:48,157 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524607967_0001_1_00_000000_0
2016-05-17 22:36:48,157 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:48,158 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:48,158 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:48,158 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:48,158 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:48,158 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:48,158 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B924C2093D174AF6BB9897202111CF88/D095F91091834CA3A0F0EDAF77477367] each/(1/1) ...platform/ALL_UNKNOWN_SWAP, VertexName: E8652D2A4D9B4B248887E2304DBD2B4F, VertexParallelism: 1, TaskAttemptID:attempt_1463524607967_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BBB86E7BE2DD498A89EBD8DC0258236D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=35A08E7CCDE24215A3CB0C6E9FCB37BF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:48,158 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:48,158 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524607967_0001_1_00_000000_0
2016-05-17 22:36:48,159 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:48,159 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:48,159 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524607967_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:48,159 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:48,161 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=35A08E7CCDE24215A3CB0C6E9FCB37BF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:48,161 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:48,161 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 35A08E7CCDE24215A3CB0C6E9FCB37BF
2016-05-17 22:36:48,161 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:48,161 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 35A08E7CCDE24215A3CB0C6E9FCB37BF
2016-05-17 22:36:48,162 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:48,161 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BBB86E7BE2DD498A89EBD8DC0258236D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:48,163 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:48,163 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BBB86E7BE2DD498A89EBD8DC0258236D
2016-05-17 22:36:48,163 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: BBB86E7BE2DD498A89EBD8DC0258236D
2016-05-17 22:36:48,166 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:48,166 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: BBB86E7BE2DD498A89EBD8DC0258236D
2016-05-17 22:36:48,166 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BBB86E7BE2DD498A89EBD8DC0258236D
2016-05-17 22:36:48,170 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E8652D2A4D9B4B248887E2304DBD2B4F, ordinal: 0
2016-05-17 22:36:48,170 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E8652D2A4D9B4B248887E2304DBD2B4F, mem on start (mb), free: 230, total: 317, max: 672
2016-05-17 22:36:48,171 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:48,171 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:48,171 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:48,175 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:48,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 35A08E7CCDE24215A3CB0C6E9FCB37BF
2016-05-17 22:36:48,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:48,176 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:48,176 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:48,176 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:48,176 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:48,176 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:48,176 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:48,176 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:48,176 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 35A08E7CCDE24215A3CB0C6E9FCB37BF, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:48,176 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BBB86E7BE2DD498A89EBD8DC0258236D, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:48,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:48,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BBB86E7BE2DD498A89EBD8DC0258236D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:48,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:48,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:48,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BBB86E7BE2DD498A89EBD8DC0258236D
2016-05-17 22:36:48,177 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:48,177 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BBB86E7BE2DD498A89EBD8DC0258236D
2016-05-17 22:36:48,177 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:48,177 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524607967_0001_1_00_000000_0
2016-05-17 22:36:48,177 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:48,178 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@72d75c44
2016-05-17 22:36:48,178 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@72d75c44
2016-05-17 22:36:48,178 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:48,178 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:48,180 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:48,180 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: BBB86E7BE2DD498A89EBD8DC0258236D
2016-05-17 22:36:48,181 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_SWAP"], id: 35A08E7CCDE24215A3CB0C6E9FCB37BF
2016-05-17 22:36:48,181 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_SWAP"]
2016-05-17 22:36:48,181 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:48,181 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:48,182 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E8652D2A4D9B4B248887E2304DBD2B4F, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:48,185 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:48,185 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:48,185 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524607967_0001_1_00_000000_0
2016-05-17 22:36:48,185 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524607967_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:48,186 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524607967_0001_1_00_000000_0
2016-05-17 22:36:48,186 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E8652D2A4D9B4B248887E2304DBD2B4F, mem on close (mb), free: 228, total: 317, max: 672
2016-05-17 22:36:48 Completed running task attempt: attempt_1463524607967_0001_1_00_000000_0
2016-05-17 22:36:48,187 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524607967_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:36:48,187 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=332398592, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524608182, Process_Duration=4, Process_End_Time=1463524608186, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:36:48,187 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607967_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E8652D2A4D9B4B248887E2304DBD2B4F, taskAttemptId=attempt_1463524607967_0001_1_00_000000_0, startTime=1463524608156, finishTime=1463524608187, timeTaken=31, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=332398592, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524608182, Process_Duration=4, Process_End_Time=1463524608186, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:48,187 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:48,188 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524607967_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:36:48,188 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:36:48,188 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:48,188 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:48,188 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524607967_0001_00_000001
2016-05-17 22:36:48,188 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524607967_0001_1_00_000000_0
2016-05-17 22:36:48,188 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:48,189 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607967_0001_1][Event:TASK_FINISHED]: vertexName=E8652D2A4D9B4B248887E2304DBD2B4F, taskId=task_1463524607967_0001_1_00_000000, startTime=1463524608156, finishTime=1463524608188, timeTaken=32, status=SUCCEEDED, successfulAttemptID=attempt_1463524607967_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=332398592, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524608182, Process_Duration=4, Process_End_Time=1463524608186, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:48,189 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524607967_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:36:48,189 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F] : 1
2016-05-17 22:36:48,189 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F]
2016-05-17 22:36:48,190 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:48,190 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607967_0001_1][Event:VERTEX_FINISHED]: vertexName=E8652D2A4D9B4B248887E2304DBD2B4F, vertexId=vertex_1463524607967_0001_1_00, initRequestedTime=1463524608125, initedTime=1463524608146, startRequestedTime=1463524608146, startedTime=1463524608146, finishTime=1463524608189, timeTaken=43, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=332398592, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524608182, Process_Duration=4, Process_End_Time=1463524608186, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524608156, firstTasksToStart=[ task_1463524607967_0001_1_00_000000 ], lastTaskFinishTime=1463524608187, lastTasksToFinish=[ task_1463524607967_0001_1_00_000000 ], minTaskDuration=31, maxTaskDuration=31, avgTaskDuration=31.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524607967_0001_1_00_000000 ], longestDurationTasks=[ task_1463524607967_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:36:48,190 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:36:48,190 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524607967_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:36:48,190 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607967_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524607967_0001_00_000001, stoppedTime=1463524608190, exitStatus=0
2016-05-17 22:36:48,190 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524607967_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:48,190 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:48,190 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:36:48,190 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524607967_0001_1
2016-05-17 22:36:48,191 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607967_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524607967_0001_1
2016-05-17 22:36:48,191 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 35A08E7CCDE24215A3CB0C6E9FCB37BF for vertex: vertex_1463524607967_0001_1_00 [E8652D2A4D9B4B248887E2304DBD2B4F]
2016-05-17 22:36:48,192 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524607967_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:36:48,192 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524607967_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:36:48,192 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524607967_0001_00_000001, Exiting
2016-05-17 22:36:48,192 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524607967_0001_00_000001
2016-05-17 22:36:48,192 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524607967_0001_00_000001
2016-05-17 22:36:48,193 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524607967_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:36:48,193 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524607967_0001_00_000001 completed successfully
2016-05-17 22:36:48,200 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524607967_0001_1, queueSize=0
2016-05-17 22:36:48,200 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:48,201 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524607967_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524607967_0001_1, startTime=1463524608125, finishTime=1463524608199, timeTaken=74, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=332398592, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524608182, Process_Duration=4, Process_End_Time=1463524608186, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:36:48,201 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524607967_0001_1 finished with state: SUCCEEDED
2016-05-17 22:36:48,202 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524607967_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:48,202 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524607967_0001_00_000001
2016-05-17 22:36:48,202 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524607967_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:48 Completed Dag: dag_1463524607967_0001_1
2016-05-17 22:36:48,203 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524607967_0001_1, dagState=SUCCEEDED
2016-05-17 22:36:48,203 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524607967_0001_1_post
2016-05-17 22:36:48,203 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:48,203 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:48,207 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/ALL_UNKNOWN_SWAP, applicationId=application_1463524607967_0001
2016-05-17 22:36:48,209 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:48,209 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:48,209 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:48,209 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:48,209 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:48,209 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:48,209 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:48,210 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:48,210 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:48,210 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:48,210 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:48,210 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:48,210 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:48,214 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B924C2093D174AF6BB9897202111CF88/D095F91091834CA3A0F0EDAF77477367/.tez/application_1463524607967_0001
2016-05-17 22:36:48,214 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_UNKNOWN_SWAP/_temporary
2016-05-17 22:36:48,215 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.264
2016-05-17 22:36:48,264 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:48,264 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:48,276 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_UNKNOWN_SWAP
2016-05-17 22:36:48,297 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:48,297 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:48,325 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-05-17 22:36:48,329 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 22:36:48,330 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:48,330 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:48,330 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:48,344 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:48,345 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_-1
2016-05-17 22:36:48,352 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:48,352 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:48,352 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_-1"]
2016-05-17 22:36:48,353 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:48,353 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:48,353 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:48,353 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...blyplatform/ALL_VALUES_-1
2016-05-17 22:36:48,353 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:48,386 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:48,390 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:48,391 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:48,391 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:48,401 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/137C89194BDC48FBBE9FB827D620CEE5/AF47A11322044335A4A5E7AEA04492A8/.tez/application_1463524608391_0001 doesn't exist and is created
2016-05-17 22:36:48,417 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:48,417 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:48,417 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/137C89194BDC48FBBE9FB827D620CEE5/AF47A11322044335A4A5E7AEA04492A8/.tez/application_1463524608391_0001
2016-05-17 22:36:48,419 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524608391_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:48,429 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:48,430 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:48,430 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:48,431 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:48,431 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/137C89194BDC48FBBE9FB827D620CEE5/AF47A11322044335A4A5E7AEA04492A8/.tez/application_1463524608391_0001/localmode-log-dir
2016-05-17 22:36:48,431 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/137C89194BDC48FBBE9FB827D620CEE5/AF47A11322044335A4A5E7AEA04492A8/.tez/application_1463524608391_0001/localmode-log-dir/history.txt.appattempt_1463524608391_0001_000000, maxErrors=10
2016-05-17 22:36:48,431 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:48,431 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524608391_0001_000000, appSubmitTime=1463524608419, launchTime=1463524608419
2016-05-17 22:36:48,433 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:48,434 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:48,435 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:48,437 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:43087
2016-05-17 22:36:48,438 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:48,439 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:48,440 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524608391_0001_000000, startTime=1463524608440
2016-05-17 22:36:48,440 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:48,440 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:48,517 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:48,517 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:48,517 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/ALL_VALUES_-1, applicationId=application_1463524608391_0001, dagName=[137C89194BDC48FBBE9FB827D620CEE5/AF47A11322044335A4A5E7AEA04492A8] each/(1/1) ...blyplatform/ALL_VALUES_-1
2016-05-17 22:36:48,530 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [137C89194BDC48FBBE9FB827D620CEE5/AF47A11322044335A4A5E7AEA04492A8] each/(1/1) ...blyplatform/ALL_VALUES_-1
2016-05-17 22:36:48,532 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524608391_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/137C89194BDC48FBBE9FB827D620CEE5/AF47A11322044335A4A5E7AEA04492A8/.tez/application_1463524608391_0001/localmode-log-dir/dag_1463524608391_0001_1.dot
2016-05-17 22:36:48,532 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/137C89194BDC48FBBE9FB827D620CEE5/AF47A11322044335A4A5E7AEA04492A8/.tez/application_1463524608391_0001/localmode-log-dir/dag_1463524608391_0001_1-tez-dag.pb.txt
2016-05-17 22:36:48,536 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524608391_0001_1
2016-05-17 22:36:48,536 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:48,537 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [137C89194BDC48FBBE9FB827D620CEE5/AF47A11322044335A4A5E7AEA04492A8] each/(1/1) ...blyplatform/ALL_VALUES_-1
2016-05-17 22:36:48 Running Dag: dag_1463524608391_0001_1
2016-05-17 22:36:48,548 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608391_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524608391_0001_1, submitTime=1463524608530
2016-05-17 22:36:48,548 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:48,549 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FCCC5EDFFB0C43649A2EF7E398109DB7
2016-05-17 22:36:48,549 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:48,549 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608391_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524608391_0001_1, initTime=1463524608548
2016-05-17 22:36:48,549 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:48,550 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524608391_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:48,550 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608391_0001_1][Event:DAG_STARTED]: dagID=dag_1463524608391_0001_1, startTime=1463524608550
2016-05-17 22:36:48,550 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:48,550 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524608391_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:48,550 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FCCC5EDFFB0C43649A2EF7E398109DB7 : {4A344B919ABD43529540B64F98D29AFE={InputName=4A344B919ABD43529540B64F98D29AFE}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:48,551 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 4A344B919ABD43529540B64F98D29AFE, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:48,551 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7]
2016-05-17 22:36:48,551 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7]
2016-05-17 22:36:48,551 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7]
2016-05-17 22:36:48,551 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7]. Starting root input initializers: 1
2016-05-17 22:36:48,551 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:48,552 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:48,552 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 4A344B919ABD43529540B64F98D29AFE on vertex vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7]
2016-05-17 22:36:48,552 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/ALL_VALUES_-1, applicationId=application_1463524608391_0001, dagName=[137C89194BDC48FBBE9FB827D620CEE5/AF47A11322044335A4A5E7AEA04492A8] each/(1/1) ...blyplatform/ALL_VALUES_-1
2016-05-17 22:36:48,553 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:48,553 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 4A344B919ABD43529540B64F98D29AFE asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:48,554 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:48,554 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:48,555 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:48,555 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 4A344B919ABD43529540B64F98D29AFE on vertex vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7]
2016-05-17 22:36:48,555 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:48,555 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {4A344B919ABD43529540B64F98D29AFE=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:48,556 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7] parallelism set to 1
2016-05-17 22:36:48,556 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:48,556 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608391_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524608391_0001_1_00, eventCount=1
2016-05-17 22:36:48,556 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7]
2016-05-17 22:36:48,556 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=9F8056D6BF4845CBADF03489C87471DE, vertexId=vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:48,556 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=9F8056D6BF4845CBADF03489C87471DE, vertexId=vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7]
2016-05-17 22:36:48,557 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=9F8056D6BF4845CBADF03489C87471DE, vertexName=FCCC5EDFFB0C43649A2EF7E398109DB7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:48,557 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=9F8056D6BF4845CBADF03489C87471DE, vertexId=vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7]
2016-05-17 22:36:48,558 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608391_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FCCC5EDFFB0C43649A2EF7E398109DB7, vertexId=vertex_1463524608391_0001_1_00, initRequestedTime=1463524608550, initedTime=1463524608558, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:48,559 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7] with distanceFromRoot: 0
2016-05-17 22:36:48,558 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:48,559 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:48,559 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FCCC5EDFFB0C43649A2EF7E398109DB7
2016-05-17 22:36:48,559 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7]
2016-05-17 22:36:48,559 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:48,559 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608391_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524608391_0001_1_00, startRequestedTime=1463524608559, startedTime=1463524608559
2016-05-17 22:36:48,560 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:48,560 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:48,560 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608391_0001_1][Event:TASK_STARTED]: vertexName=FCCC5EDFFB0C43649A2EF7E398109DB7, taskId=task_1463524608391_0001_1_00_000000, scheduledTime=1463524608560, launchTime=1463524608560
2016-05-17 22:36:48,560 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524608391_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:48,560 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7]
2016-05-17 22:36:48,560 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [137C89194BDC48FBBE9FB827D620CEE5/AF47A11322044335A4A5E7AEA04492A8] each/(1/1) ...blyplatform/ALL_VALUES_-1, VertexName: FCCC5EDFFB0C43649A2EF7E398109DB7, VertexParallelism: 1, TaskAttemptID:attempt_1463524608391_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4A344B919ABD43529540B64F98D29AFE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=9F8056D6BF4845CBADF03489C87471DE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:48,560 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524608391_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:48,561 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:48,561 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:48,562 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:48,562 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:48,563 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524608391_0001_00_000001
2016-05-17 22:36:48,563 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524608391_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:48,565 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524608391_0001_00_000001
2016-05-17 22:36:48,565 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:48,569 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524608391_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:48,569 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:48,569 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608391_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524608391_0001_00_000001, launchTime=1463524608569
2016-05-17 22:36:48,571 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524608391_0001_00_000001
2016-05-17 22:36:48,571 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524608391_0001_1_00_000000_0] to container: [container_1463524608391_0001_00_000001]
2016-05-17 22:36:48,571 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524608391_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:48,572 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524608391_0001_00_000001 given task: attempt_1463524608391_0001_1_00_000000_0
2016-05-17 22:36:48,572 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524608391_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524608391_0001_1_00_000000_0
2016-05-17 22:36:48,572 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524608391_0001_1_00_000000_0] started. Is using containerId: [container_1463524608391_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:48,572 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:48 Starting to run new task attempt: attempt_1463524608391_0001_1_00_000000_0
2016-05-17 22:36:48,572 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524608391_0001_1_00_000000_0
2016-05-17 22:36:48,572 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:48,572 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608391_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FCCC5EDFFB0C43649A2EF7E398109DB7, taskAttemptId=attempt_1463524608391_0001_1_00_000000_0, startTime=1463524608572, containerId=container_1463524608391_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524608391_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:48,572 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:48,572 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524608391_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:48,573 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524608391_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:48,572 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:48,574 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:48,574 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:48,574 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:48,574 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [137C89194BDC48FBBE9FB827D620CEE5/AF47A11322044335A4A5E7AEA04492A8] each/(1/1) ...blyplatform/ALL_VALUES_-1, VertexName: FCCC5EDFFB0C43649A2EF7E398109DB7, VertexParallelism: 1, TaskAttemptID:attempt_1463524608391_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4A344B919ABD43529540B64F98D29AFE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=9F8056D6BF4845CBADF03489C87471DE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:48,575 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:48,575 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524608391_0001, with dag id: dag_1463524608391_0001_1
2016-05-17 22:36:48,575 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524608391_0001_1_00_000000_0
2016-05-17 22:36:48,575 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:48,576 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:48,576 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=4A344B919ABD43529540B64F98D29AFE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:48,577 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:48,577 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 4A344B919ABD43529540B64F98D29AFE
2016-05-17 22:36:48,577 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 4A344B919ABD43529540B64F98D29AFE
2016-05-17 22:36:48,577 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=9F8056D6BF4845CBADF03489C87471DE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:48,576 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524608391_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:48,576 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:48,578 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:48,579 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 9F8056D6BF4845CBADF03489C87471DE
2016-05-17 22:36:48,579 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:48,579 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 9F8056D6BF4845CBADF03489C87471DE
2016-05-17 22:36:48,581 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:48,582 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:48,582 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 4A344B919ABD43529540B64F98D29AFE
2016-05-17 22:36:48,583 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 4A344B919ABD43529540B64F98D29AFE
2016-05-17 22:36:48,587 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FCCC5EDFFB0C43649A2EF7E398109DB7, ordinal: 0
2016-05-17 22:36:48,587 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FCCC5EDFFB0C43649A2EF7E398109DB7, mem on start (mb), free: 153, total: 317, max: 672
2016-05-17 22:36:48,587 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:48,587 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:48,587 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:48,591 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:48,591 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 9F8056D6BF4845CBADF03489C87471DE
2016-05-17 22:36:48,591 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:48,591 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:48,591 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:48,592 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:48,592 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:48,592 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:48,592 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:48,592 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:48,592 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 4A344B919ABD43529540B64F98D29AFE, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:48,593 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 9F8056D6BF4845CBADF03489C87471DE, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:48,593 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:48,593 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 4A344B919ABD43529540B64F98D29AFE
2016-05-17 22:36:48,593 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:48,593 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 4A344B919ABD43529540B64F98D29AFE
2016-05-17 22:36:48,593 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 4A344B919ABD43529540B64F98D29AFE being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:48,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:48,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:48,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:48,594 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524608391_0001_1_00_000000_0
2016-05-17 22:36:48,594 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:48,595 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1544f0d4
2016-05-17 22:36:48,595 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1544f0d4
2016-05-17 22:36:48,596 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:48,596 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:48,597 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:48,597 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 4A344B919ABD43529540B64F98D29AFE
2016-05-17 22:36:48,598 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_-1"], id: 9F8056D6BF4845CBADF03489C87471DE
2016-05-17 22:36:48,598 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_-1"]
2016-05-17 22:36:48,598 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:48,598 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:48,598 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FCCC5EDFFB0C43649A2EF7E398109DB7, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:48,599 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:48,599 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:48,600 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:48,600 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:48,601 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524608391_0001_1_00_000000_0
2016-05-17 22:36:48,601 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524608391_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:48,601 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524608391_0001_1_00_000000_0
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:48,603 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=332398592, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524608598, Process_Duration=3, Process_End_Time=1463524608601, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:36:48,603 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:48,603 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608391_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FCCC5EDFFB0C43649A2EF7E398109DB7, taskAttemptId=attempt_1463524608391_0001_1_00_000000_0, startTime=1463524608572, finishTime=1463524608603, timeTaken=31, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=332398592, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524608598, Process_Duration=3, Process_End_Time=1463524608601, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:36:48,603 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524608391_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:36:48,603 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524608391_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:36:48,604 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608391_0001_1][Event:TASK_FINISHED]: vertexName=FCCC5EDFFB0C43649A2EF7E398109DB7, taskId=task_1463524608391_0001_1_00_000000, startTime=1463524608572, finishTime=1463524608603, timeTaken=31, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
], counters=Counters: 0
2016-05-17 22:36:48,604 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524608391_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:36:48,604 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7] : 1
2016-05-17 22:36:48,604 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:48,604 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:48,603 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FCCC5EDFFB0C43649A2EF7E398109DB7, mem on close (mb), free: 151, total: 317, max: 672
2016-05-17 22:36:48,604 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7] because task failed: task_1463524608391_0001_1_00_000000
2016-05-17 22:36:48,604 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:36:48,604 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:36:48,604 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7]
2016-05-17 22:36:48,605 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=9F8056D6BF4845CBADF03489C87471DE, vertexId=vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7]
2016-05-17 22:36:48,606 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:48,606 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608391_0001_1][Event:VERTEX_FINISHED]: vertexName=FCCC5EDFFB0C43649A2EF7E398109DB7, vertexId=vertex_1463524608391_0001_1_00, initRequestedTime=1463524608550, initedTime=1463524608558, startRequestedTime=1463524608559, startedTime=1463524608559, finishTime=1463524608606, timeTaken=47, status=FAILED, diagnostics=Task failed, taskId=task_1463524608391_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524608572, firstTasksToStart=[ task_1463524608391_0001_1_00_000000 ], lastTaskFinishTime=1463524608603, lastTasksToFinish=[ task_1463524608391_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:36:48,606 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:36:48,606 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608391_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524608391_0001_00_000001, stoppedTime=1463524608606, exitStatus=0
2016-05-17 22:36:48,606 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524608391_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:48,606 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524608391_0001_1_00_000000_0 failed: true container: container_1463524608391_0001_00_000001 numFailedTAs: 0
2016-05-17 22:36:48,606 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:48,607 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=FCCC5EDFFB0C43649A2EF7E398109DB7, index=0
2016-05-17 22:36:48,607 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=FCCC5EDFFB0C43649A2EF7E398109DB7, sourceVertex=4A344B919ABD43529540B64F98D29AFE
2016-05-17 22:36:48,607 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=FCCC5EDFFB0C43649A2EF7E398109DB7, sourceVertex=9F8056D6BF4845CBADF03489C87471DE
2016-05-17 22:36:48,607 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:36:48,607 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:36:48,607 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524608391_0001_00_000001
2016-05-17 22:36:48,607 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524608391_0001_00_000001
2016-05-17 22:36:48,607 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524608391_0001_1
2016-05-17 22:36:48,608 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 9F8056D6BF4845CBADF03489C87471DE for vertex: vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7]
2016-05-17 22:36:48,609 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524608391_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:36:48,609 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524608391_0001_1, queueSize=0
2016-05-17 22:36:48,609 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:48,609 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608391_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524608391_0001_1, startTime=1463524608550, finishTime=1463524608608, timeTaken=58, status=FAILED, diagnostics=Vertex failed, vertexName=FCCC5EDFFB0C43649A2EF7E398109DB7, vertexId=vertex_1463524608391_0001_1_00, diagnostics=[Task failed, taskId=task_1463524608391_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524608391_0001_1_00 [FCCC5EDFFB0C43649A2EF7E398109DB7] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:36:48,609 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524608391_0001_1 finished with state: FAILED
2016-05-17 22:36:48,609 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524608391_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:48,610 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524608391_0001_00_000001
2016-05-17 22:36:48 Completed Dag: dag_1463524608391_0001_1
2016-05-17 22:36:48,609 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524608391_0001_00_000001 completed but with errors
2016-05-17 22:36:48,610 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524608391_0001_1, dagState=FAILED
2016-05-17 22:36:48,610 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524608391_0001_1_post
2016-05-17 22:36:48,610 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:48,610 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:48,611 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524608391_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:36:48,611 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524608391_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:36:48,611 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524608391_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:36:48,611 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524608391_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:36:48,613 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/ALL_VALUES_-1, applicationId=application_1463524608391_0001
2016-05-17 22:36:48,614 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:48,614 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:48,614 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:48,614 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:48,614 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:48,616 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:48,616 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:48,617 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:48,617 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:48,617 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:48,617 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:48,617 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:48,617 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:48,621 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/137C89194BDC48FBBE9FB827D620CEE5/AF47A11322044335A4A5E7AEA04492A8/.tez/application_1463524608391_0001
2016-05-17 22:36:48,621 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping all jobs
2016-05-17 22:36:48,621 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping: (1/1) ...blyplatform/ALL_VALUES_-1
2016-05-17 22:36:48,622 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopped all jobs
2016-05-17 22:36:48,622 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_-1/_temporary
2016-05-17 22:36:48,622 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.270
2016-05-17 22:36:48,622 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_VALUES_-1
2016-05-17 22:36:48,652 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:48,653 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:48,679 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.002
2016-05-17 22:36:48,682 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-05-17 22:36:48,682 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:48,682 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:48,682 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_VALUES_'field'
2016-05-17 22:36:48,701 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:48,701 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:48,731 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-05-17 22:36:48,731 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-05-17 22:36:48,732 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:48,732 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:48,732 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:48,734 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:48,734 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:48,744 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:48,745 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_RESULTS
2016-05-17 22:36:48,751 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:48,752 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:48,752 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_RESULTS"]
2016-05-17 22:36:48,752 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:48,752 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:48,752 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:48,753 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...atform/ALL_VALUES_RESULTS
2016-05-17 22:36:48,753 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:48,765 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:48,769 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:48,769 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:48,769 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:48,778 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C1E8AE893BD245388B7A06E95417F93C/C7AEA895CDCD45BE8DB4DBEB255EFBE9/.tez/application_1463524608769_0001 doesn't exist and is created
2016-05-17 22:36:48,793 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:48,794 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:48,794 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C1E8AE893BD245388B7A06E95417F93C/C7AEA895CDCD45BE8DB4DBEB255EFBE9/.tez/application_1463524608769_0001
2016-05-17 22:36:48,796 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524608769_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:48,806 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:48,807 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:48,808 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:48,808 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:48,808 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C1E8AE893BD245388B7A06E95417F93C/C7AEA895CDCD45BE8DB4DBEB255EFBE9/.tez/application_1463524608769_0001/localmode-log-dir
2016-05-17 22:36:48,808 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C1E8AE893BD245388B7A06E95417F93C/C7AEA895CDCD45BE8DB4DBEB255EFBE9/.tez/application_1463524608769_0001/localmode-log-dir/history.txt.appattempt_1463524608769_0001_000000, maxErrors=10
2016-05-17 22:36:48,808 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:48,809 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524608769_0001_000000, appSubmitTime=1463524608796, launchTime=1463524608796
2016-05-17 22:36:48,814 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:48,814 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:48,814 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:58620
2016-05-17 22:36:48,815 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:48,817 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:48,819 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524608769_0001_000000, startTime=1463524608818
2016-05-17 22:36:48,819 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:48,819 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:48,819 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:48,894 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:48,894 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:48,894 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/ALL_VALUES_RESULTS, applicationId=application_1463524608769_0001, dagName=[C1E8AE893BD245388B7A06E95417F93C/C7AEA895CDCD45BE8DB4DBEB255EFBE9] each/(1/1) ...atform/ALL_VALUES_RESULTS
2016-05-17 22:36:48,907 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [C1E8AE893BD245388B7A06E95417F93C/C7AEA895CDCD45BE8DB4DBEB255EFBE9] each/(1/1) ...atform/ALL_VALUES_RESULTS
2016-05-17 22:36:48,909 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524608769_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C1E8AE893BD245388B7A06E95417F93C/C7AEA895CDCD45BE8DB4DBEB255EFBE9/.tez/application_1463524608769_0001/localmode-log-dir/dag_1463524608769_0001_1.dot
2016-05-17 22:36:48,909 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C1E8AE893BD245388B7A06E95417F93C/C7AEA895CDCD45BE8DB4DBEB255EFBE9/.tez/application_1463524608769_0001/localmode-log-dir/dag_1463524608769_0001_1-tez-dag.pb.txt
2016-05-17 22:36:48,913 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524608769_0001_1
2016-05-17 22:36:48,913 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:48,913 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [C1E8AE893BD245388B7A06E95417F93C/C7AEA895CDCD45BE8DB4DBEB255EFBE9] each/(1/1) ...atform/ALL_VALUES_RESULTS
2016-05-17 22:36:48 Running Dag: dag_1463524608769_0001_1
2016-05-17 22:36:48,929 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608769_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524608769_0001_1, submitTime=1463524608907
2016-05-17 22:36:48,929 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:48,929 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FB804E5EB1594D559077578DAF10A3D6
2016-05-17 22:36:48,930 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:48,930 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608769_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524608769_0001_1, initTime=1463524608929
2016-05-17 22:36:48,930 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:48,930 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524608769_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:48,931 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608769_0001_1][Event:DAG_STARTED]: dagID=dag_1463524608769_0001_1, startTime=1463524608931
2016-05-17 22:36:48,931 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:48,931 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:48,931 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524608769_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:48,931 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FB804E5EB1594D559077578DAF10A3D6 : {629AA866EB994D069EC3BB9206B5E339={InputName=629AA866EB994D069EC3BB9206B5E339}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:48,931 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 629AA866EB994D069EC3BB9206B5E339, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:48,931 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6]
2016-05-17 22:36:48,931 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6]
2016-05-17 22:36:48,931 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6]
2016-05-17 22:36:48,931 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6]. Starting root input initializers: 1
2016-05-17 22:36:48,932 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:48,932 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 629AA866EB994D069EC3BB9206B5E339 on vertex vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6]
2016-05-17 22:36:48,932 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/ALL_VALUES_RESULTS, applicationId=application_1463524608769_0001, dagName=[C1E8AE893BD245388B7A06E95417F93C/C7AEA895CDCD45BE8DB4DBEB255EFBE9] each/(1/1) ...atform/ALL_VALUES_RESULTS
2016-05-17 22:36:48,934 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:48,934 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 629AA866EB994D069EC3BB9206B5E339 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:48,934 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:48,935 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:48,936 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:48,936 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 629AA866EB994D069EC3BB9206B5E339 on vertex vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6]
2016-05-17 22:36:48,936 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:48,936 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {629AA866EB994D069EC3BB9206B5E339=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:48,936 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6] parallelism set to 1
2016-05-17 22:36:48,937 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:48,937 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608769_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524608769_0001_1_00, eventCount=1
2016-05-17 22:36:48,937 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6]
2016-05-17 22:36:48,937 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A28C2F1E70424BCCB15143A25BBBA595, vertexId=vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:48,937 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A28C2F1E70424BCCB15143A25BBBA595, vertexId=vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6]
2016-05-17 22:36:48,939 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A28C2F1E70424BCCB15143A25BBBA595, vertexName=FB804E5EB1594D559077578DAF10A3D6, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:48,939 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A28C2F1E70424BCCB15143A25BBBA595, vertexId=vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6]
2016-05-17 22:36:48,952 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:48,952 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608769_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FB804E5EB1594D559077578DAF10A3D6, vertexId=vertex_1463524608769_0001_1_00, initRequestedTime=1463524608931, initedTime=1463524608952, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:48,953 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6] with distanceFromRoot: 0
2016-05-17 22:36:48,953 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:48,953 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FB804E5EB1594D559077578DAF10A3D6
2016-05-17 22:36:48,953 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6]
2016-05-17 22:36:48,954 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:48,954 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608769_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524608769_0001_1_00, startRequestedTime=1463524608953, startedTime=1463524608953
2016-05-17 22:36:48,954 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:48,955 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:48,955 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608769_0001_1][Event:TASK_STARTED]: vertexName=FB804E5EB1594D559077578DAF10A3D6, taskId=task_1463524608769_0001_1_00_000000, scheduledTime=1463524608954, launchTime=1463524608954
2016-05-17 22:36:48,955 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524608769_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:48,955 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6]
2016-05-17 22:36:48,956 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C1E8AE893BD245388B7A06E95417F93C/C7AEA895CDCD45BE8DB4DBEB255EFBE9] each/(1/1) ...atform/ALL_VALUES_RESULTS, VertexName: FB804E5EB1594D559077578DAF10A3D6, VertexParallelism: 1, TaskAttemptID:attempt_1463524608769_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=629AA866EB994D069EC3BB9206B5E339, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A28C2F1E70424BCCB15143A25BBBA595, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:48,956 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524608769_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:48,956 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:48,957 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:48,957 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:48,957 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:48,958 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524608769_0001_00_000001
2016-05-17 22:36:48,958 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524608769_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:48,959 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524608769_0001_00_000001
2016-05-17 22:36:48,959 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:48,960 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:48,960 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524608769_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:48,960 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608769_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524608769_0001_00_000001, launchTime=1463524608960
2016-05-17 22:36:48,961 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524608769_0001_00_000001
2016-05-17 22:36:48,961 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524608769_0001_1_00_000000_0] to container: [container_1463524608769_0001_00_000001]
2016-05-17 22:36:48,961 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524608769_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:48,961 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524608769_0001_00_000001 given task: attempt_1463524608769_0001_1_00_000000_0
2016-05-17 22:36:48,961 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524608769_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524608769_0001_1_00_000000_0
2016-05-17 22:36:48,962 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524608769_0001_1_00_000000_0] started. Is using containerId: [container_1463524608769_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:48 Starting to run new task attempt: attempt_1463524608769_0001_1_00_000000_0
2016-05-17 22:36:48,962 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524608769_0001_1_00_000000_0
2016-05-17 22:36:48,962 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:48,962 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:48,962 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:48,962 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:48,962 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:48,962 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:48,962 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:48,962 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C1E8AE893BD245388B7A06E95417F93C/C7AEA895CDCD45BE8DB4DBEB255EFBE9] each/(1/1) ...atform/ALL_VALUES_RESULTS, VertexName: FB804E5EB1594D559077578DAF10A3D6, VertexParallelism: 1, TaskAttemptID:attempt_1463524608769_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=629AA866EB994D069EC3BB9206B5E339, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A28C2F1E70424BCCB15143A25BBBA595, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:48,963 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:48,963 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524608769_0001_1_00_000000_0
2016-05-17 22:36:48,963 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:48,963 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:48,964 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:48,962 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608769_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FB804E5EB1594D559077578DAF10A3D6, taskAttemptId=attempt_1463524608769_0001_1_00_000000_0, startTime=1463524608961, containerId=container_1463524608769_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524608769_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:48,965 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524608769_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:48,965 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524608769_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:48,966 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524608769_0001, with dag id: dag_1463524608769_0001_1
2016-05-17 22:36:48,967 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:48,968 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=629AA866EB994D069EC3BB9206B5E339, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:48,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:48,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 629AA866EB994D069EC3BB9206B5E339
2016-05-17 22:36:48,969 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524608769_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:48,969 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 629AA866EB994D069EC3BB9206B5E339
2016-05-17 22:36:48,970 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:48,971 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 629AA866EB994D069EC3BB9206B5E339
2016-05-17 22:36:48,971 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 629AA866EB994D069EC3BB9206B5E339
2016-05-17 22:36:48,971 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A28C2F1E70424BCCB15143A25BBBA595, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:48,972 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:48,972 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A28C2F1E70424BCCB15143A25BBBA595
2016-05-17 22:36:48,973 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:48,973 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A28C2F1E70424BCCB15143A25BBBA595
2016-05-17 22:36:48,973 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FB804E5EB1594D559077578DAF10A3D6, ordinal: 0
2016-05-17 22:36:48,974 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FB804E5EB1594D559077578DAF10A3D6, mem on start (mb), free: 226, total: 313, max: 672
2016-05-17 22:36:48,974 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:48,974 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:48,974 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:48,981 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:48,981 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A28C2F1E70424BCCB15143A25BBBA595
2016-05-17 22:36:48,981 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:48,982 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:48,982 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:48,983 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:48,983 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:48,983 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:48,983 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:48,983 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:48,983 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 629AA866EB994D069EC3BB9206B5E339, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:48,984 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A28C2F1E70424BCCB15143A25BBBA595, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:48,984 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:48,984 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 629AA866EB994D069EC3BB9206B5E339
2016-05-17 22:36:48,984 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:48,984 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 629AA866EB994D069EC3BB9206B5E339
2016-05-17 22:36:48,984 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 629AA866EB994D069EC3BB9206B5E339 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:48,984 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:48,985 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:48,985 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:48,985 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524608769_0001_1_00_000000_0
2016-05-17 22:36:48,985 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:48,986 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@62374a0
2016-05-17 22:36:48,986 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@62374a0
2016-05-17 22:36:48,987 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:48,987 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:48,988 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:48,988 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 629AA866EB994D069EC3BB9206B5E339
2016-05-17 22:36:48,988 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_RESULTS"], id: A28C2F1E70424BCCB15143A25BBBA595
2016-05-17 22:36:48,989 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_RESULTS"]
2016-05-17 22:36:48,989 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:48,989 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:48,989 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FB804E5EB1594D559077578DAF10A3D6, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:48,990 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:48,992 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:48,994 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:48,994 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:48,995 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524608769_0001_1_00_000000_0
2016-05-17 22:36:48,995 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524608769_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:48,996 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524608769_0001_1_00_000000_0
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:48,999 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=328204288, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524608990, Process_Duration=6, Process_End_Time=1463524608996, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:36:48,999 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608769_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FB804E5EB1594D559077578DAF10A3D6, taskAttemptId=attempt_1463524608769_0001_1_00_000000_0, startTime=1463524608961, finishTime=1463524608999, timeTaken=38, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=328204288, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524608990, Process_Duration=6, Process_End_Time=1463524608996, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:36:48,999 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:49,000 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524608769_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:36:48,999 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FB804E5EB1594D559077578DAF10A3D6, mem on close (mb), free: 223, total: 313, max: 672
2016-05-17 22:36:49,000 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524608769_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:36:49,000 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=FB804E5EB1594D559077578DAF10A3D6, index=0
2016-05-17 22:36:49,000 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:49,000 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=FB804E5EB1594D559077578DAF10A3D6, sourceVertex=629AA866EB994D069EC3BB9206B5E339
2016-05-17 22:36:49,000 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608769_0001_1][Event:TASK_FINISHED]: vertexName=FB804E5EB1594D559077578DAF10A3D6, taskId=task_1463524608769_0001_1_00_000000, startTime=1463524608961, finishTime=1463524609000, timeTaken=39, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
], counters=Counters: 0
2016-05-17 22:36:49,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=FB804E5EB1594D559077578DAF10A3D6, sourceVertex=A28C2F1E70424BCCB15143A25BBBA595
2016-05-17 22:36:49,001 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524608769_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:36:49,001 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524608769_0001_00_000001
2016-05-17 22:36:49,001 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524608769_0001_00_000001
2016-05-17 22:36:49,001 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6] : 1
2016-05-17 22:36:49,001 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6] because task failed: task_1463524608769_0001_1_00_000000
2016-05-17 22:36:49,001 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:36:49,001 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:36:49,001 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6]
2016-05-17 22:36:49,001 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:49,001 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=A28C2F1E70424BCCB15143A25BBBA595, vertexId=vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6]
2016-05-17 22:36:49,002 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524608769_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:36:49,002 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524608769_0001_00_000001 completed but with errors
2016-05-17 22:36:49,003 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:49,003 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608769_0001_1][Event:VERTEX_FINISHED]: vertexName=FB804E5EB1594D559077578DAF10A3D6, vertexId=vertex_1463524608769_0001_1_00, initRequestedTime=1463524608931, initedTime=1463524608952, startRequestedTime=1463524608953, startedTime=1463524608953, finishTime=1463524609003, timeTaken=50, status=FAILED, diagnostics=Task failed, taskId=task_1463524608769_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524608961, firstTasksToStart=[ task_1463524608769_0001_1_00_000000 ], lastTaskFinishTime=1463524608999, lastTasksToFinish=[ task_1463524608769_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:36:49,003 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:36:49,003 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608769_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524608769_0001_00_000001, stoppedTime=1463524609003, exitStatus=0
2016-05-17 22:36:49,003 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524608769_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:49,003 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524608769_0001_1_00_000000_0 failed: true container: container_1463524608769_0001_00_000001 numFailedTAs: 0
2016-05-17 22:36:49,003 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:49,003 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524608769_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:36:49,004 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524608769_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:49,004 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:36:49,004 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:36:49,004 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524608769_0001_1
2016-05-17 22:36:49,004 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: A28C2F1E70424BCCB15143A25BBBA595 for vertex: vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6]
2016-05-17 22:36:49,005 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524608769_0001_1, queueSize=0
2016-05-17 22:36:49,006 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:49,006 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524608769_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524608769_0001_1, startTime=1463524608931, finishTime=1463524609004, timeTaken=73, status=FAILED, diagnostics=Vertex failed, vertexName=FB804E5EB1594D559077578DAF10A3D6, vertexId=vertex_1463524608769_0001_1_00, diagnostics=[Task failed, taskId=task_1463524608769_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524608769_0001_1_00 [FB804E5EB1594D559077578DAF10A3D6] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:36:49,006 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524608769_0001_1 finished with state: FAILED
2016-05-17 22:36:49,006 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524608769_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:49,006 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524608769_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:36:49,006 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524608769_0001_00_000001
2016-05-17 22:36:49,010 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/ALL_VALUES_RESULTS, applicationId=application_1463524608769_0001
2016-05-17 22:36:49 Completed Dag: dag_1463524608769_0001_1
2016-05-17 22:36:49,010 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524608769_0001_1, dagState=FAILED
2016-05-17 22:36:49,010 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524608769_0001_1_post
2016-05-17 22:36:49,010 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:49,010 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:49,013 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:49,013 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:49,013 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:49,013 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:49,013 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:49,014 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:49,014 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:49,014 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:49,015 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:49,015 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:49,015 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:49,015 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:49,015 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:49,018 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C1E8AE893BD245388B7A06E95417F93C/C7AEA895CDCD45BE8DB4DBEB255EFBE9/.tez/application_1463524608769_0001
2016-05-17 22:36:49,018 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping all jobs
2016-05-17 22:36:49,018 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping: (1/1) ...atform/ALL_VALUES_RESULTS
2016-05-17 22:36:49,019 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopped all jobs
2016-05-17 22:36:49,019 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_RESULTS/_temporary
2016-05-17 22:36:49,019 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.268
2016-05-17 22:36:49,019 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_VALUES_RESULTS
2016-05-17 22:36:49,047 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:49,047 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:49,077 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-17 22:36:49,082 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 22:36:49,082 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:49,082 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:49,082 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:49,096 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:49,097 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_ALL
2016-05-17 22:36:49,105 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:49,105 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:49,105 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_ALL"]
2016-05-17 22:36:49,106 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:49,106 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:49,106 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:49,107 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...lyplatform/ALL_VALUES_ALL
2016-05-17 22:36:49,107 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:49,118 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:49,122 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:49,122 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:49,122 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:49,131 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/873C69D346EF4FB08216307BB8948097/E474E77B7F8648648A35025B62041098/.tez/application_1463524609122_0001 doesn't exist and is created
2016-05-17 22:36:49,146 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:49,146 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:49,147 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/873C69D346EF4FB08216307BB8948097/E474E77B7F8648648A35025B62041098/.tez/application_1463524609122_0001
2016-05-17 22:36:49,149 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524609122_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:49,156 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:49,156 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:49,159 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:49,159 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:49,160 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:49,160 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:49,160 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/873C69D346EF4FB08216307BB8948097/E474E77B7F8648648A35025B62041098/.tez/application_1463524609122_0001/localmode-log-dir
2016-05-17 22:36:49,160 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/873C69D346EF4FB08216307BB8948097/E474E77B7F8648648A35025B62041098/.tez/application_1463524609122_0001/localmode-log-dir/history.txt.appattempt_1463524609122_0001_000000, maxErrors=10
2016-05-17 22:36:49,161 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:49,161 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524609122_0001_000000, appSubmitTime=1463524609149, launchTime=1463524609149
2016-05-17 22:36:49,165 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:55651
2016-05-17 22:36:49,165 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:49,166 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:49,167 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:49,168 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:49,170 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:49,170 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524609122_0001_000000, startTime=1463524609170
2016-05-17 22:36:49,170 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:49,170 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:49,247 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:49,247 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:49,247 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_VALUES_ALL, applicationId=application_1463524609122_0001, dagName=[873C69D346EF4FB08216307BB8948097/E474E77B7F8648648A35025B62041098] each/(1/1) ...lyplatform/ALL_VALUES_ALL
2016-05-17 22:36:49,260 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [873C69D346EF4FB08216307BB8948097/E474E77B7F8648648A35025B62041098] each/(1/1) ...lyplatform/ALL_VALUES_ALL
2016-05-17 22:36:49,261 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524609122_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/873C69D346EF4FB08216307BB8948097/E474E77B7F8648648A35025B62041098/.tez/application_1463524609122_0001/localmode-log-dir/dag_1463524609122_0001_1.dot
2016-05-17 22:36:49,262 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/873C69D346EF4FB08216307BB8948097/E474E77B7F8648648A35025B62041098/.tez/application_1463524609122_0001/localmode-log-dir/dag_1463524609122_0001_1-tez-dag.pb.txt
2016-05-17 22:36:49,265 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524609122_0001_1
2016-05-17 22:36:49,265 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:49,265 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [873C69D346EF4FB08216307BB8948097/E474E77B7F8648648A35025B62041098] each/(1/1) ...lyplatform/ALL_VALUES_ALL
2016-05-17 22:36:49 Running Dag: dag_1463524609122_0001_1
2016-05-17 22:36:49,276 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609122_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524609122_0001_1, submitTime=1463524609260
2016-05-17 22:36:49,276 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:49,276 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FF866356E4E249AB8C3973A7EDA368F5
2016-05-17 22:36:49,276 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:49,277 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609122_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524609122_0001_1, initTime=1463524609276
2016-05-17 22:36:49,277 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:49,277 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524609122_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:49,277 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609122_0001_1][Event:DAG_STARTED]: dagID=dag_1463524609122_0001_1, startTime=1463524609277
2016-05-17 22:36:49,277 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:49,277 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524609122_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:49,277 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FF866356E4E249AB8C3973A7EDA368F5 : {E510A090959C4259B59F21C498533BB3={InputName=E510A090959C4259B59F21C498533BB3}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:49,278 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: E510A090959C4259B59F21C498533BB3, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:49,278 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5]
2016-05-17 22:36:49,278 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5]
2016-05-17 22:36:49,277 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:49,278 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5]
2016-05-17 22:36:49,278 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5]. Starting root input initializers: 1
2016-05-17 22:36:49,279 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:49,279 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: E510A090959C4259B59F21C498533BB3 on vertex vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5]
2016-05-17 22:36:49,279 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_VALUES_ALL, applicationId=application_1463524609122_0001, dagName=[873C69D346EF4FB08216307BB8948097/E474E77B7F8648648A35025B62041098] each/(1/1) ...lyplatform/ALL_VALUES_ALL
2016-05-17 22:36:49,281 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:49,281 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input E510A090959C4259B59F21C498533BB3 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:49,281 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:49,282 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:49,283 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:49,283 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: E510A090959C4259B59F21C498533BB3 on vertex vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5]
2016-05-17 22:36:49,283 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:49,284 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {E510A090959C4259B59F21C498533BB3=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:49,284 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5] parallelism set to 1
2016-05-17 22:36:49,284 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:49,284 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609122_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524609122_0001_1_00, eventCount=1
2016-05-17 22:36:49,284 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5]
2016-05-17 22:36:49,284 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=DA1A9BB2402E41DC9CF8CF2C0546CD67, vertexId=vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:49,284 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=DA1A9BB2402E41DC9CF8CF2C0546CD67, vertexId=vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5]
2016-05-17 22:36:49,286 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=DA1A9BB2402E41DC9CF8CF2C0546CD67, vertexName=FF866356E4E249AB8C3973A7EDA368F5, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:49,287 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=DA1A9BB2402E41DC9CF8CF2C0546CD67, vertexId=vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5]
2016-05-17 22:36:49,288 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609122_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FF866356E4E249AB8C3973A7EDA368F5, vertexId=vertex_1463524609122_0001_1_00, initRequestedTime=1463524609277, initedTime=1463524609287, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:49,288 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5] with distanceFromRoot: 0
2016-05-17 22:36:49,288 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:49,288 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:49,288 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FF866356E4E249AB8C3973A7EDA368F5
2016-05-17 22:36:49,288 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5]
2016-05-17 22:36:49,288 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:49,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609122_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524609122_0001_1_00, startRequestedTime=1463524609288, startedTime=1463524609288
2016-05-17 22:36:49,289 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:49,289 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:49,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609122_0001_1][Event:TASK_STARTED]: vertexName=FF866356E4E249AB8C3973A7EDA368F5, taskId=task_1463524609122_0001_1_00_000000, scheduledTime=1463524609289, launchTime=1463524609289
2016-05-17 22:36:49,289 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524609122_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:49,289 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5]
2016-05-17 22:36:49,290 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [873C69D346EF4FB08216307BB8948097/E474E77B7F8648648A35025B62041098] each/(1/1) ...lyplatform/ALL_VALUES_ALL, VertexName: FF866356E4E249AB8C3973A7EDA368F5, VertexParallelism: 1, TaskAttemptID:attempt_1463524609122_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E510A090959C4259B59F21C498533BB3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DA1A9BB2402E41DC9CF8CF2C0546CD67, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:49,290 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524609122_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:49,290 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:49,290 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:49,291 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:49,291 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:49,291 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524609122_0001_00_000001
2016-05-17 22:36:49,291 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524609122_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:49,292 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524609122_0001_00_000001
2016-05-17 22:36:49,292 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:49,293 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524609122_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:49,293 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:49,293 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609122_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524609122_0001_00_000001, launchTime=1463524609293
2016-05-17 22:36:49,294 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524609122_0001_00_000001
2016-05-17 22:36:49,294 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524609122_0001_1_00_000000_0] to container: [container_1463524609122_0001_00_000001]
2016-05-17 22:36:49,294 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524609122_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:49,294 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524609122_0001_00_000001 given task: attempt_1463524609122_0001_1_00_000000_0
2016-05-17 22:36:49,294 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524609122_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524609122_0001_1_00_000000_0
2016-05-17 22:36:49,294 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524609122_0001_1_00_000000_0] started. Is using containerId: [container_1463524609122_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:49,295 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609122_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FF866356E4E249AB8C3973A7EDA368F5, taskAttemptId=attempt_1463524609122_0001_1_00_000000_0, startTime=1463524609294, containerId=container_1463524609122_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524609122_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:49,295 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524609122_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:49,295 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524609122_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:49,295 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:49 Starting to run new task attempt: attempt_1463524609122_0001_1_00_000000_0
2016-05-17 22:36:49,295 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524609122_0001_1_00_000000_0
2016-05-17 22:36:49,295 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:49,296 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:49,296 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:49,296 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:49,296 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:49,296 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:49,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [873C69D346EF4FB08216307BB8948097/E474E77B7F8648648A35025B62041098] each/(1/1) ...lyplatform/ALL_VALUES_ALL, VertexName: FF866356E4E249AB8C3973A7EDA368F5, VertexParallelism: 1, TaskAttemptID:attempt_1463524609122_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E510A090959C4259B59F21C498533BB3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DA1A9BB2402E41DC9CF8CF2C0546CD67, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:49,296 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:49,297 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524609122_0001_1_00_000000_0
2016-05-17 22:36:49,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:49,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:49,297 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524609122_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:49,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:49,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DA1A9BB2402E41DC9CF8CF2C0546CD67, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:49,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:49,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DA1A9BB2402E41DC9CF8CF2C0546CD67
2016-05-17 22:36:49,298 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:49,299 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: DA1A9BB2402E41DC9CF8CF2C0546CD67
2016-05-17 22:36:49,300 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:49,305 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E510A090959C4259B59F21C498533BB3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:49,305 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:49,305 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E510A090959C4259B59F21C498533BB3
2016-05-17 22:36:49,306 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: E510A090959C4259B59F21C498533BB3
2016-05-17 22:36:49,308 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:49,309 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:49,309 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DA1A9BB2402E41DC9CF8CF2C0546CD67
2016-05-17 22:36:49,309 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: E510A090959C4259B59F21C498533BB3
2016-05-17 22:36:49,310 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E510A090959C4259B59F21C498533BB3
2016-05-17 22:36:49,309 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FF866356E4E249AB8C3973A7EDA368F5, ordinal: 0
2016-05-17 22:36:49,310 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF866356E4E249AB8C3973A7EDA368F5, mem on start (mb), free: 164, total: 313, max: 672
2016-05-17 22:36:49,310 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:49,310 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:49,310 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:49,310 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:49,309 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524609122_0001, with dag id: dag_1463524609122_0001_1
2016-05-17 22:36:49,311 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:49,311 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:49,311 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:49,311 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:49,311 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:49,311 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:49,311 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:49,312 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DA1A9BB2402E41DC9CF8CF2C0546CD67, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:49,312 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E510A090959C4259B59F21C498533BB3, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:49,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:49,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E510A090959C4259B59F21C498533BB3 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:49,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E510A090959C4259B59F21C498533BB3
2016-05-17 22:36:49,312 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:49,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E510A090959C4259B59F21C498533BB3
2016-05-17 22:36:49,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:49,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:49,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:49,313 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524609122_0001_1_00_000000_0
2016-05-17 22:36:49,314 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:49,314 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4829a51e
2016-05-17 22:36:49,315 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4829a51e
2016-05-17 22:36:49,315 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:49,316 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:49,316 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:49,316 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: E510A090959C4259B59F21C498533BB3
2016-05-17 22:36:49,316 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_ALL"], id: DA1A9BB2402E41DC9CF8CF2C0546CD67
2016-05-17 22:36:49,317 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_ALL"]
2016-05-17 22:36:49,317 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:49,317 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:49,317 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FF866356E4E249AB8C3973A7EDA368F5, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:49,318 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:49,318 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:49,323 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:49,323 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:49,323 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524609122_0001_1_00_000000_0
2016-05-17 22:36:49,323 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524609122_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:49,324 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524609122_0001_1_00_000000_0
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:49,327 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=328204288, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524609317, Process_Duration=7, Process_End_Time=1463524609324, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:36:49,327 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF866356E4E249AB8C3973A7EDA368F5, mem on close (mb), free: 162, total: 313, max: 672
2016-05-17 22:36:49,328 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609122_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FF866356E4E249AB8C3973A7EDA368F5, taskAttemptId=attempt_1463524609122_0001_1_00_000000_0, startTime=1463524609294, finishTime=1463524609327, timeTaken=33, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=328204288, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524609317, Process_Duration=7, Process_End_Time=1463524609324, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:36:49,328 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524609122_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:36:49,328 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:49,328 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524609122_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:36:49,328 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609122_0001_1][Event:TASK_FINISHED]: vertexName=FF866356E4E249AB8C3973A7EDA368F5, taskId=task_1463524609122_0001_1_00_000000, startTime=1463524609294, finishTime=1463524609328, timeTaken=34, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
], counters=Counters: 0
2016-05-17 22:36:49,328 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524609122_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:36:49,329 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5] : 1
2016-05-17 22:36:49,329 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:49,329 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:49,329 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5] because task failed: task_1463524609122_0001_1_00_000000
2016-05-17 22:36:49,329 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:36:49,329 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:36:49,329 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5]
2016-05-17 22:36:49,330 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=DA1A9BB2402E41DC9CF8CF2C0546CD67, vertexId=vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5]
2016-05-17 22:36:49,329 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=FF866356E4E249AB8C3973A7EDA368F5, index=0
2016-05-17 22:36:49,330 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=FF866356E4E249AB8C3973A7EDA368F5, sourceVertex=E510A090959C4259B59F21C498533BB3
2016-05-17 22:36:49,330 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=FF866356E4E249AB8C3973A7EDA368F5, sourceVertex=DA1A9BB2402E41DC9CF8CF2C0546CD67
2016-05-17 22:36:49,330 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524609122_0001_00_000001
2016-05-17 22:36:49,330 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524609122_0001_00_000001
2016-05-17 22:36:49,331 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609122_0001_1][Event:VERTEX_FINISHED]: vertexName=FF866356E4E249AB8C3973A7EDA368F5, vertexId=vertex_1463524609122_0001_1_00, initRequestedTime=1463524609277, initedTime=1463524609287, startRequestedTime=1463524609288, startedTime=1463524609288, finishTime=1463524609331, timeTaken=43, status=FAILED, diagnostics=Task failed, taskId=task_1463524609122_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524609294, firstTasksToStart=[ task_1463524609122_0001_1_00_000000 ], lastTaskFinishTime=1463524609327, lastTasksToFinish=[ task_1463524609122_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:36:49,331 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:49,331 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:36:49,332 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609122_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524609122_0001_00_000001, stoppedTime=1463524609332, exitStatus=0
2016-05-17 22:36:49,332 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524609122_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:49,332 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524609122_0001_1_00_000000_0 failed: true container: container_1463524609122_0001_00_000001 numFailedTAs: 0
2016-05-17 22:36:49,332 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:36:49,332 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:36:49,332 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524609122_0001_1
2016-05-17 22:36:49,332 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: DA1A9BB2402E41DC9CF8CF2C0546CD67 for vertex: vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5]
2016-05-17 22:36:49,333 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524609122_0001_1, queueSize=0
2016-05-17 22:36:49,333 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609122_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524609122_0001_1, startTime=1463524609277, finishTime=1463524609332, timeTaken=55, status=FAILED, diagnostics=Vertex failed, vertexName=FF866356E4E249AB8C3973A7EDA368F5, vertexId=vertex_1463524609122_0001_1_00, diagnostics=[Task failed, taskId=task_1463524609122_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524609122_0001_1_00 [FF866356E4E249AB8C3973A7EDA368F5] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:36:49,334 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524609122_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:36:49,333 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:49,335 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524609122_0001_00_000001 completed but with errors
2016-05-17 22:36:49,335 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:49,335 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524609122_0001_1 finished with state: FAILED
2016-05-17 22:36:49,335 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524609122_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:49,336 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524609122_0001_00_000001
2016-05-17 22:36:49 Completed Dag: dag_1463524609122_0001_1
2016-05-17 22:36:49,337 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524609122_0001_1, dagState=FAILED
2016-05-17 22:36:49,338 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524609122_0001_1_post
2016-05-17 22:36:49,338 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:49,339 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:49,339 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524609122_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:36:49,339 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/ALL_VALUES_ALL, applicationId=application_1463524609122_0001
2016-05-17 22:36:49,339 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524609122_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:49,339 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524609122_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:36:49,340 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:49,341 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:49,341 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:49,341 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:49,341 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:49,341 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:49,341 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:49,341 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:49,342 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:49,342 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:49,342 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:49,342 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:49,342 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:49,345 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/873C69D346EF4FB08216307BB8948097/E474E77B7F8648648A35025B62041098/.tez/application_1463524609122_0001
2016-05-17 22:36:49,345 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping all jobs
2016-05-17 22:36:49,345 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping: (1/1) ...lyplatform/ALL_VALUES_ALL
2016-05-17 22:36:49,346 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopped all jobs
2016-05-17 22:36:49,346 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_ALL/_temporary
2016-05-17 22:36:49,346 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.241
2016-05-17 22:36:49,346 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_VALUES_ALL
2016-05-17 22:36:49,366 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:49,367 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:49,394 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-17 22:36:49,399 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 22:36:49,399 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:49,399 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:49,400 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:49,412 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:49,413 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_REPLACE
2016-05-17 22:36:49,420 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:49,420 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:49,420 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_REPLACE"]
2016-05-17 22:36:49,420 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:49,421 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:49,421 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:49,421 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...atform/ALL_VALUES_REPLACE
2016-05-17 22:36:49,421 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:49,432 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:49,436 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:49,437 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:49,437 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:49,446 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2BB2D559691A492B8DDC797A866DE86B/EBF15BDAEAAB49B3A0EC6687E17C51EC/.tez/application_1463524609436_0001 doesn't exist and is created
2016-05-17 22:36:49,462 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:49,462 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:49,463 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2BB2D559691A492B8DDC797A866DE86B/EBF15BDAEAAB49B3A0EC6687E17C51EC/.tez/application_1463524609436_0001
2016-05-17 22:36:49,465 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524609436_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:49,476 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:49,476 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:49,477 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:49,477 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:49,477 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2BB2D559691A492B8DDC797A866DE86B/EBF15BDAEAAB49B3A0EC6687E17C51EC/.tez/application_1463524609436_0001/localmode-log-dir
2016-05-17 22:36:49,477 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2BB2D559691A492B8DDC797A866DE86B/EBF15BDAEAAB49B3A0EC6687E17C51EC/.tez/application_1463524609436_0001/localmode-log-dir/history.txt.appattempt_1463524609436_0001_000000, maxErrors=10
2016-05-17 22:36:49,477 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:49,478 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524609436_0001_000000, appSubmitTime=1463524609464, launchTime=1463524609465
2016-05-17 22:36:49,482 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:49,482 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:49,482 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:37474
2016-05-17 22:36:49,483 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:49,483 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:49,484 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:49,485 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524609436_0001_000000, startTime=1463524609485
2016-05-17 22:36:49,485 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:49,485 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:49,563 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:49,563 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:49,563 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/ALL_VALUES_REPLACE, applicationId=application_1463524609436_0001, dagName=[2BB2D559691A492B8DDC797A866DE86B/EBF15BDAEAAB49B3A0EC6687E17C51EC] each/(1/1) ...atform/ALL_VALUES_REPLACE
2016-05-17 22:36:49,568 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:49,568 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:49,577 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [2BB2D559691A492B8DDC797A866DE86B/EBF15BDAEAAB49B3A0EC6687E17C51EC] each/(1/1) ...atform/ALL_VALUES_REPLACE
2016-05-17 22:36:49,578 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524609436_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2BB2D559691A492B8DDC797A866DE86B/EBF15BDAEAAB49B3A0EC6687E17C51EC/.tez/application_1463524609436_0001/localmode-log-dir/dag_1463524609436_0001_1.dot
2016-05-17 22:36:49,578 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2BB2D559691A492B8DDC797A866DE86B/EBF15BDAEAAB49B3A0EC6687E17C51EC/.tez/application_1463524609436_0001/localmode-log-dir/dag_1463524609436_0001_1-tez-dag.pb.txt
2016-05-17 22:36:49,582 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524609436_0001_1
2016-05-17 22:36:49,582 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:49,582 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [2BB2D559691A492B8DDC797A866DE86B/EBF15BDAEAAB49B3A0EC6687E17C51EC] each/(1/1) ...atform/ALL_VALUES_REPLACE
2016-05-17 22:36:49 Running Dag: dag_1463524609436_0001_1
2016-05-17 22:36:49,593 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:49,594 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609436_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524609436_0001_1, submitTime=1463524609577
2016-05-17 22:36:49,594 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A2AF84748DDD4319B4A8F319D65F47E4
2016-05-17 22:36:49,594 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:49,594 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:49,595 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609436_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524609436_0001_1, initTime=1463524609594
2016-05-17 22:36:49,595 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524609436_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:49,595 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609436_0001_1][Event:DAG_STARTED]: dagID=dag_1463524609436_0001_1, startTime=1463524609595
2016-05-17 22:36:49,595 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:49,595 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524609436_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:49,595 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:49,595 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A2AF84748DDD4319B4A8F319D65F47E4 : {5E97B6BCFAAE47ABB8DB1979B058F5D4={InputName=5E97B6BCFAAE47ABB8DB1979B058F5D4}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:49,596 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 5E97B6BCFAAE47ABB8DB1979B058F5D4, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:49,596 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4]
2016-05-17 22:36:49,595 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/ALL_VALUES_REPLACE, applicationId=application_1463524609436_0001, dagName=[2BB2D559691A492B8DDC797A866DE86B/EBF15BDAEAAB49B3A0EC6687E17C51EC] each/(1/1) ...atform/ALL_VALUES_REPLACE
2016-05-17 22:36:49,596 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4]
2016-05-17 22:36:49,596 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4]
2016-05-17 22:36:49,596 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4]. Starting root input initializers: 1
2016-05-17 22:36:49,596 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:49,597 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 5E97B6BCFAAE47ABB8DB1979B058F5D4 on vertex vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4]
2016-05-17 22:36:49,599 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:49,600 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 5E97B6BCFAAE47ABB8DB1979B058F5D4 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:49,600 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:49,601 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:49,602 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:49,603 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 5E97B6BCFAAE47ABB8DB1979B058F5D4 on vertex vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4]
2016-05-17 22:36:49,603 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:49,603 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {5E97B6BCFAAE47ABB8DB1979B058F5D4=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:49,603 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4] parallelism set to 1
2016-05-17 22:36:49,604 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:49,604 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609436_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524609436_0001_1_00, eventCount=1
2016-05-17 22:36:49,604 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4]
2016-05-17 22:36:49,604 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D7B636D51AD845C69700B689BE52F551, vertexId=vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:49,604 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D7B636D51AD845C69700B689BE52F551, vertexId=vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4]
2016-05-17 22:36:49,606 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D7B636D51AD845C69700B689BE52F551, vertexName=A2AF84748DDD4319B4A8F319D65F47E4, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:49,606 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D7B636D51AD845C69700B689BE52F551, vertexId=vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4]
2016-05-17 22:36:49,607 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609436_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A2AF84748DDD4319B4A8F319D65F47E4, vertexId=vertex_1463524609436_0001_1_00, initRequestedTime=1463524609595, initedTime=1463524609607, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:49,607 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:49,608 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4] with distanceFromRoot: 0
2016-05-17 22:36:49,609 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:49,610 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A2AF84748DDD4319B4A8F319D65F47E4
2016-05-17 22:36:49,610 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4]
2016-05-17 22:36:49,610 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609436_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524609436_0001_1_00, startRequestedTime=1463524609609, startedTime=1463524609609
2016-05-17 22:36:49,610 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:49,611 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:49,612 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609436_0001_1][Event:TASK_STARTED]: vertexName=A2AF84748DDD4319B4A8F319D65F47E4, taskId=task_1463524609436_0001_1_00_000000, scheduledTime=1463524609611, launchTime=1463524609611
2016-05-17 22:36:49,612 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524609436_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:49,612 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:49,612 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4]
2016-05-17 22:36:49,613 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2BB2D559691A492B8DDC797A866DE86B/EBF15BDAEAAB49B3A0EC6687E17C51EC] each/(1/1) ...atform/ALL_VALUES_REPLACE, VertexName: A2AF84748DDD4319B4A8F319D65F47E4, VertexParallelism: 1, TaskAttemptID:attempt_1463524609436_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5E97B6BCFAAE47ABB8DB1979B058F5D4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D7B636D51AD845C69700B689BE52F551, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:49,614 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524609436_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:49,614 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:49,615 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:49,615 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:49,615 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:49,616 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524609436_0001_00_000001
2016-05-17 22:36:49,616 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524609436_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:49,616 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524609436_0001, with dag id: dag_1463524609436_0001_1
2016-05-17 22:36:49,617 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524609436_0001_00_000001
2016-05-17 22:36:49,618 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:49,622 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524609436_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:49,622 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:49,622 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609436_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524609436_0001_00_000001, launchTime=1463524609621
2016-05-17 22:36:49,622 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524609436_0001_00_000001
2016-05-17 22:36:49,623 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524609436_0001_1_00_000000_0] to container: [container_1463524609436_0001_00_000001]
2016-05-17 22:36:49,623 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524609436_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:49,623 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524609436_0001_1_00_000000_0] started. Is using containerId: [container_1463524609436_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:49,623 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609436_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A2AF84748DDD4319B4A8F319D65F47E4, taskAttemptId=attempt_1463524609436_0001_1_00_000000_0, startTime=1463524609623, containerId=container_1463524609436_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524609436_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:49,624 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524609436_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:49,624 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524609436_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:49,624 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:49,624 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524609436_0001_00_000001 given task: attempt_1463524609436_0001_1_00_000000_0
2016-05-17 22:36:49,624 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524609436_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524609436_0001_1_00_000000_0
2016-05-17 22:36:49 Starting to run new task attempt: attempt_1463524609436_0001_1_00_000000_0
2016-05-17 22:36:49,625 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524609436_0001_1_00_000000_0
2016-05-17 22:36:49,625 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:49,625 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:49,625 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:49,625 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:49,625 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:49,625 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:49,625 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2BB2D559691A492B8DDC797A866DE86B/EBF15BDAEAAB49B3A0EC6687E17C51EC] each/(1/1) ...atform/ALL_VALUES_REPLACE, VertexName: A2AF84748DDD4319B4A8F319D65F47E4, VertexParallelism: 1, TaskAttemptID:attempt_1463524609436_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5E97B6BCFAAE47ABB8DB1979B058F5D4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D7B636D51AD845C69700B689BE52F551, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:49,626 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:49,626 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524609436_0001_1_00_000000_0
2016-05-17 22:36:49,626 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:49,626 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:49,627 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524609436_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:49,627 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=5E97B6BCFAAE47ABB8DB1979B058F5D4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:49,627 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:49,628 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 5E97B6BCFAAE47ABB8DB1979B058F5D4
2016-05-17 22:36:49,628 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 5E97B6BCFAAE47ABB8DB1979B058F5D4
2016-05-17 22:36:49,628 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:49,630 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:49,630 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 5E97B6BCFAAE47ABB8DB1979B058F5D4
2016-05-17 22:36:49,630 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 5E97B6BCFAAE47ABB8DB1979B058F5D4
2016-05-17 22:36:49,630 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D7B636D51AD845C69700B689BE52F551, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:49,631 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:49,631 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D7B636D51AD845C69700B689BE52F551
2016-05-17 22:36:49,631 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:49,631 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D7B636D51AD845C69700B689BE52F551
2016-05-17 22:36:49,632 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:49,639 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A2AF84748DDD4319B4A8F319D65F47E4, ordinal: 0
2016-05-17 22:36:49,639 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A2AF84748DDD4319B4A8F319D65F47E4, mem on start (mb), free: 105, total: 313, max: 672
2016-05-17 22:36:49,639 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:49,640 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:49,640 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:49,645 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:49,645 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D7B636D51AD845C69700B689BE52F551
2016-05-17 22:36:49,645 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:49,645 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:49,645 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:49,645 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:49,646 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:49,646 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:49,646 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:49,646 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:49,646 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 5E97B6BCFAAE47ABB8DB1979B058F5D4, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:49,646 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D7B636D51AD845C69700B689BE52F551, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:49,646 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:49,646 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 5E97B6BCFAAE47ABB8DB1979B058F5D4 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:49,646 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 5E97B6BCFAAE47ABB8DB1979B058F5D4
2016-05-17 22:36:49,646 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:49,646 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 5E97B6BCFAAE47ABB8DB1979B058F5D4
2016-05-17 22:36:49,647 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:49,647 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:49,647 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:49,647 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524609436_0001_1_00_000000_0
2016-05-17 22:36:49,647 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:49,648 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@50cc3bf1
2016-05-17 22:36:49,648 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@50cc3bf1
2016-05-17 22:36:49,648 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:49,648 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:49,650 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:49,650 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 5E97B6BCFAAE47ABB8DB1979B058F5D4
2016-05-17 22:36:49,651 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_REPLACE"], id: D7B636D51AD845C69700B689BE52F551
2016-05-17 22:36:49,651 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_REPLACE"]
2016-05-17 22:36:49,651 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:49,651 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:49,651 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A2AF84748DDD4319B4A8F319D65F47E4, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:49,652 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:49,652 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:49,652 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:49,652 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:49,653 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524609436_0001_1_00_000000_0
2016-05-17 22:36:49,653 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524609436_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:49,653 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524609436_0001_1_00_000000_0
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:49,654 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=328204288, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524609651, Process_Duration=2, Process_End_Time=1463524609653, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:36:49,655 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A2AF84748DDD4319B4A8F319D65F47E4, mem on close (mb), free: 103, total: 313, max: 672
2016-05-17 22:36:49,655 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=A2AF84748DDD4319B4A8F319D65F47E4, index=0
2016-05-17 22:36:49,655 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=A2AF84748DDD4319B4A8F319D65F47E4, sourceVertex=5E97B6BCFAAE47ABB8DB1979B058F5D4
2016-05-17 22:36:49,655 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:49,655 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609436_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A2AF84748DDD4319B4A8F319D65F47E4, taskAttemptId=attempt_1463524609436_0001_1_00_000000_0, startTime=1463524609623, finishTime=1463524609654, timeTaken=31, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=328204288, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524609651, Process_Duration=2, Process_End_Time=1463524609653, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:36:49,655 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524609436_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:36:49,655 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524609436_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:36:49,655 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:49,655 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=A2AF84748DDD4319B4A8F319D65F47E4, sourceVertex=D7B636D51AD845C69700B689BE52F551
2016-05-17 22:36:49,655 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609436_0001_1][Event:TASK_FINISHED]: vertexName=A2AF84748DDD4319B4A8F319D65F47E4, taskId=task_1463524609436_0001_1_00_000000, startTime=1463524609623, finishTime=1463524609655, timeTaken=32, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
], counters=Counters: 0
2016-05-17 22:36:49,656 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524609436_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:36:49,656 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4] : 1
2016-05-17 22:36:49,656 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4] because task failed: task_1463524609436_0001_1_00_000000
2016-05-17 22:36:49,656 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:36:49,656 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:36:49,656 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4]
2016-05-17 22:36:49,656 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=D7B636D51AD845C69700B689BE52F551, vertexId=vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4]
2016-05-17 22:36:49,657 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:49,658 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:49,658 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609436_0001_1][Event:VERTEX_FINISHED]: vertexName=A2AF84748DDD4319B4A8F319D65F47E4, vertexId=vertex_1463524609436_0001_1_00, initRequestedTime=1463524609595, initedTime=1463524609607, startRequestedTime=1463524609609, startedTime=1463524609609, finishTime=1463524609657, timeTaken=48, status=FAILED, diagnostics=Task failed, taskId=task_1463524609436_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524609623, firstTasksToStart=[ task_1463524609436_0001_1_00_000000 ], lastTaskFinishTime=1463524609654, lastTasksToFinish=[ task_1463524609436_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:36:49,658 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524609436_0001_00_000001
2016-05-17 22:36:49,658 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524609436_0001_00_000001
2016-05-17 22:36:49,658 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:36:49,658 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609436_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524609436_0001_00_000001, stoppedTime=1463524609658, exitStatus=0
2016-05-17 22:36:49,658 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524609436_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:49,658 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524609436_0001_1_00_000000_0 failed: true container: container_1463524609436_0001_00_000001 numFailedTAs: 0
2016-05-17 22:36:49,658 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:36:49,659 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:36:49,659 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524609436_0001_1
2016-05-17 22:36:49,659 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: D7B636D51AD845C69700B689BE52F551 for vertex: vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4]
2016-05-17 22:36:49,659 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:49,659 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524609436_0001_1, queueSize=0
2016-05-17 22:36:49,660 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:49,660 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524609436_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:36:49,660 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524609436_0001_00_000001 completed but with errors
2016-05-17 22:36:49,660 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609436_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524609436_0001_1, startTime=1463524609595, finishTime=1463524609659, timeTaken=64, status=FAILED, diagnostics=Vertex failed, vertexName=A2AF84748DDD4319B4A8F319D65F47E4, vertexId=vertex_1463524609436_0001_1_00, diagnostics=[Task failed, taskId=task_1463524609436_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524609436_0001_1_00 [A2AF84748DDD4319B4A8F319D65F47E4] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:36:49,660 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524609436_0001_1 finished with state: FAILED
2016-05-17 22:36:49,661 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524609436_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:49,661 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524609436_0001_00_000001
2016-05-17 22:36:49,661 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524609436_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:36:49,662 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524609436_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:49 Completed Dag: dag_1463524609436_0001_1
2016-05-17 22:36:49,662 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524609436_0001_1, dagState=FAILED
2016-05-17 22:36:49,662 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524609436_0001_1_post
2016-05-17 22:36:49,662 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:49,662 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:49,662 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524609436_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:36:49,665 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/ALL_VALUES_REPLACE, applicationId=application_1463524609436_0001
2016-05-17 22:36:49,666 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:49,666 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:49,666 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:49,666 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:49,666 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:49,667 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:49,667 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:49,667 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:49,667 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:49,668 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:49,668 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:49,668 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:49,668 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:49,671 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2BB2D559691A492B8DDC797A866DE86B/EBF15BDAEAAB49B3A0EC6687E17C51EC/.tez/application_1463524609436_0001
2016-05-17 22:36:49,671 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping all jobs
2016-05-17 22:36:49,672 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping: (1/1) ...atform/ALL_VALUES_REPLACE
2016-05-17 22:36:49,672 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopped all jobs
2016-05-17 22:36:49,672 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_REPLACE/_temporary
2016-05-17 22:36:49,672 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.252
2016-05-17 22:36:49,673 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_VALUES_REPLACE
2016-05-17 22:36:49,695 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:49,695 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:49,722 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-05-17 22:36:49,730 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 22:36:49,730 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:49,731 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:49,731 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:49,757 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:49,757 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_SWAP
2016-05-17 22:36:49,764 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:49,764 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:49,765 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_SWAP"]
2016-05-17 22:36:49,765 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:49,765 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:49,765 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:49,765 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...yplatform/ALL_VALUES_SWAP
2016-05-17 22:36:49,765 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:49,776 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:49,780 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:49,781 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:49,781 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:49,790 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2BB0E167975A4437A042A5F3E565E491/C410CCBB368D40FE9249DBA717248602/.tez/application_1463524609781_0001 doesn't exist and is created
2016-05-17 22:36:49,806 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:49,806 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:49,806 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2BB0E167975A4437A042A5F3E565E491/C410CCBB368D40FE9249DBA717248602/.tez/application_1463524609781_0001
2016-05-17 22:36:49,809 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524609781_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:49,819 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:49,820 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:49,821 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:49,821 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:49,821 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2BB0E167975A4437A042A5F3E565E491/C410CCBB368D40FE9249DBA717248602/.tez/application_1463524609781_0001/localmode-log-dir
2016-05-17 22:36:49,822 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2BB0E167975A4437A042A5F3E565E491/C410CCBB368D40FE9249DBA717248602/.tez/application_1463524609781_0001/localmode-log-dir/history.txt.appattempt_1463524609781_0001_000000, maxErrors=10
2016-05-17 22:36:49,822 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:49,822 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524609781_0001_000000, appSubmitTime=1463524609809, launchTime=1463524609809
2016-05-17 22:36:49,827 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:48967
2016-05-17 22:36:49,828 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:49,828 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:49,829 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:49,829 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:49,830 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:49,831 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524609781_0001_000000, startTime=1463524609831
2016-05-17 22:36:49,831 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:49,831 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:49,906 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:49,906 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:49,906 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/ALL_VALUES_SWAP, applicationId=application_1463524609781_0001, dagName=[2BB0E167975A4437A042A5F3E565E491/C410CCBB368D40FE9249DBA717248602] each/(1/1) ...yplatform/ALL_VALUES_SWAP
2016-05-17 22:36:49,919 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [2BB0E167975A4437A042A5F3E565E491/C410CCBB368D40FE9249DBA717248602] each/(1/1) ...yplatform/ALL_VALUES_SWAP
2016-05-17 22:36:49,921 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524609781_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2BB0E167975A4437A042A5F3E565E491/C410CCBB368D40FE9249DBA717248602/.tez/application_1463524609781_0001/localmode-log-dir/dag_1463524609781_0001_1.dot
2016-05-17 22:36:49,921 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2BB0E167975A4437A042A5F3E565E491/C410CCBB368D40FE9249DBA717248602/.tez/application_1463524609781_0001/localmode-log-dir/dag_1463524609781_0001_1-tez-dag.pb.txt
2016-05-17 22:36:49,924 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524609781_0001_1
2016-05-17 22:36:49,924 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:49,924 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [2BB0E167975A4437A042A5F3E565E491/C410CCBB368D40FE9249DBA717248602] each/(1/1) ...yplatform/ALL_VALUES_SWAP
2016-05-17 22:36:49 Running Dag: dag_1463524609781_0001_1
2016-05-17 22:36:49,935 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:49,936 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609781_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524609781_0001_1, submitTime=1463524609920
2016-05-17 22:36:49,936 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DF7EAD9946894C5F829999768195E515
2016-05-17 22:36:49,936 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:49,936 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:49,936 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609781_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524609781_0001_1, initTime=1463524609936
2016-05-17 22:36:49,937 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524609781_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:49,937 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609781_0001_1][Event:DAG_STARTED]: dagID=dag_1463524609781_0001_1, startTime=1463524609937
2016-05-17 22:36:49,937 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:49,937 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524609781_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:49,937 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: DF7EAD9946894C5F829999768195E515 : {51745AA8E84E4DE7B80E13C1109C24DC={InputName=51745AA8E84E4DE7B80E13C1109C24DC}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:49,937 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 51745AA8E84E4DE7B80E13C1109C24DC, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:49,937 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515]
2016-05-17 22:36:49,937 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515]
2016-05-17 22:36:49,937 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515]
2016-05-17 22:36:49,938 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515]. Starting root input initializers: 1
2016-05-17 22:36:49,938 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:49,938 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:49,938 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/ALL_VALUES_SWAP, applicationId=application_1463524609781_0001, dagName=[2BB0E167975A4437A042A5F3E565E491/C410CCBB368D40FE9249DBA717248602] each/(1/1) ...yplatform/ALL_VALUES_SWAP
2016-05-17 22:36:49,938 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 51745AA8E84E4DE7B80E13C1109C24DC on vertex vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515]
2016-05-17 22:36:49,940 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:49,941 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 51745AA8E84E4DE7B80E13C1109C24DC asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:49,941 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:49,941 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:49,942 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:49,943 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 51745AA8E84E4DE7B80E13C1109C24DC on vertex vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515]
2016-05-17 22:36:49,943 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:49,943 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {51745AA8E84E4DE7B80E13C1109C24DC=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:49,943 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515] parallelism set to 1
2016-05-17 22:36:49,943 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:49,944 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609781_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524609781_0001_1_00, eventCount=1
2016-05-17 22:36:49,944 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515]
2016-05-17 22:36:49,944 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=073B01178A324857A9F19891CE340F33, vertexId=vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:49,944 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=073B01178A324857A9F19891CE340F33, vertexId=vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515]
2016-05-17 22:36:49,946 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=073B01178A324857A9F19891CE340F33, vertexName=DF7EAD9946894C5F829999768195E515, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:49,946 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=073B01178A324857A9F19891CE340F33, vertexId=vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515]
2016-05-17 22:36:49,947 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609781_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DF7EAD9946894C5F829999768195E515, vertexId=vertex_1463524609781_0001_1_00, initRequestedTime=1463524609937, initedTime=1463524609947, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:49,947 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515] with distanceFromRoot: 0
2016-05-17 22:36:49,947 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:49,947 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:49,947 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in DF7EAD9946894C5F829999768195E515
2016-05-17 22:36:49,947 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515]
2016-05-17 22:36:49,948 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:49,948 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609781_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524609781_0001_1_00, startRequestedTime=1463524609947, startedTime=1463524609947
2016-05-17 22:36:49,948 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:49,949 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:49,949 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609781_0001_1][Event:TASK_STARTED]: vertexName=DF7EAD9946894C5F829999768195E515, taskId=task_1463524609781_0001_1_00_000000, scheduledTime=1463524609948, launchTime=1463524609948
2016-05-17 22:36:49,949 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524609781_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:49,949 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515]
2016-05-17 22:36:49,949 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2BB0E167975A4437A042A5F3E565E491/C410CCBB368D40FE9249DBA717248602] each/(1/1) ...yplatform/ALL_VALUES_SWAP, VertexName: DF7EAD9946894C5F829999768195E515, VertexParallelism: 1, TaskAttemptID:attempt_1463524609781_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=51745AA8E84E4DE7B80E13C1109C24DC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=073B01178A324857A9F19891CE340F33, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:49,949 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524609781_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:49,950 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:49,950 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:49,951 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:49,951 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:49,951 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524609781_0001_00_000001
2016-05-17 22:36:49,951 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524609781_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:49,953 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524609781_0001_00_000001
2016-05-17 22:36:49,953 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:49,954 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524609781_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:49,954 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:49,954 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609781_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524609781_0001_00_000001, launchTime=1463524609954
2016-05-17 22:36:49,955 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524609781_0001_00_000001
2016-05-17 22:36:49,956 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524609781_0001, with dag id: dag_1463524609781_0001_1
2016-05-17 22:36:49,956 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524609781_0001_1_00_000000_0] to container: [container_1463524609781_0001_00_000001]
2016-05-17 22:36:49,956 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524609781_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:49,957 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524609781_0001_1_00_000000_0] started. Is using containerId: [container_1463524609781_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:49,957 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609781_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DF7EAD9946894C5F829999768195E515, taskAttemptId=attempt_1463524609781_0001_1_00_000000_0, startTime=1463524609957, containerId=container_1463524609781_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524609781_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:49,957 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524609781_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:49,957 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:49,957 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524609781_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:49,957 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524609781_0001_00_000001 given task: attempt_1463524609781_0001_1_00_000000_0
2016-05-17 22:36:49,958 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524609781_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524609781_0001_1_00_000000_0
2016-05-17 22:36:49 Starting to run new task attempt: attempt_1463524609781_0001_1_00_000000_0
2016-05-17 22:36:49,958 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524609781_0001_1_00_000000_0
2016-05-17 22:36:49,958 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:49,959 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:49,959 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:49,959 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:49,959 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:49,959 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:49,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2BB0E167975A4437A042A5F3E565E491/C410CCBB368D40FE9249DBA717248602] each/(1/1) ...yplatform/ALL_VALUES_SWAP, VertexName: DF7EAD9946894C5F829999768195E515, VertexParallelism: 1, TaskAttemptID:attempt_1463524609781_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=51745AA8E84E4DE7B80E13C1109C24DC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=073B01178A324857A9F19891CE340F33, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:49,959 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:49,960 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524609781_0001_1_00_000000_0
2016-05-17 22:36:49,960 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:49,960 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:49,960 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524609781_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:49,961 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:49,963 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=51745AA8E84E4DE7B80E13C1109C24DC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:49,963 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:49,964 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:49,964 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 51745AA8E84E4DE7B80E13C1109C24DC
2016-05-17 22:36:49,965 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 51745AA8E84E4DE7B80E13C1109C24DC
2016-05-17 22:36:49,963 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=073B01178A324857A9F19891CE340F33, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:49,966 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:49,966 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 073B01178A324857A9F19891CE340F33
2016-05-17 22:36:49,967 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:49,967 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 073B01178A324857A9F19891CE340F33
2016-05-17 22:36:49,968 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:49,968 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 51745AA8E84E4DE7B80E13C1109C24DC
2016-05-17 22:36:49,968 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 51745AA8E84E4DE7B80E13C1109C24DC
2016-05-17 22:36:49,969 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DF7EAD9946894C5F829999768195E515, ordinal: 0
2016-05-17 22:36:49,969 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DF7EAD9946894C5F829999768195E515, mem on start (mb), free: 185, total: 305, max: 672
2016-05-17 22:36:49,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:49,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:49,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:49,975 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:49,975 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:49,977 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:49,977 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 073B01178A324857A9F19891CE340F33
2016-05-17 22:36:49,977 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:49,977 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:49,977 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:49,977 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:49,977 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:49,978 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:49,978 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:49,978 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:49,978 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 51745AA8E84E4DE7B80E13C1109C24DC, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:49,978 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 073B01178A324857A9F19891CE340F33, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:49,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:49,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 51745AA8E84E4DE7B80E13C1109C24DC being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:49,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:49,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 51745AA8E84E4DE7B80E13C1109C24DC
2016-05-17 22:36:49,978 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:49,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 51745AA8E84E4DE7B80E13C1109C24DC
2016-05-17 22:36:49,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:49,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:49,979 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524609781_0001_1_00_000000_0
2016-05-17 22:36:49,980 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:49,980 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@33b28d
2016-05-17 22:36:49,981 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:49,981 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 51745AA8E84E4DE7B80E13C1109C24DC
2016-05-17 22:36:49,981 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_SWAP"], id: 073B01178A324857A9F19891CE340F33
2016-05-17 22:36:49,981 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_SWAP"]
2016-05-17 22:36:49,982 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:49,982 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:49,982 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@33b28d
2016-05-17 22:36:49,982 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:49,982 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:49,983 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DF7EAD9946894C5F829999768195E515, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:49,983 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:49,984 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:49,985 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:49,985 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:49,985 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524609781_0001_1_00_000000_0
2016-05-17 22:36:49,986 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524609781_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:49,986 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524609781_0001_1_00_000000_0
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:49,988 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=319815680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524609983, Process_Duration=3, Process_End_Time=1463524609986, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:36:49,988 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609781_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DF7EAD9946894C5F829999768195E515, taskAttemptId=attempt_1463524609781_0001_1_00_000000_0, startTime=1463524609957, finishTime=1463524609988, timeTaken=31, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=319815680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524609983, Process_Duration=3, Process_End_Time=1463524609986, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:36:49,989 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524609781_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:36:49,989 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524609781_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:36:49,988 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:49,989 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609781_0001_1][Event:TASK_FINISHED]: vertexName=DF7EAD9946894C5F829999768195E515, taskId=task_1463524609781_0001_1_00_000000, startTime=1463524609957, finishTime=1463524609989, timeTaken=32, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
], counters=Counters: 0
2016-05-17 22:36:49,988 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DF7EAD9946894C5F829999768195E515, mem on close (mb), free: 182, total: 305, max: 672
2016-05-17 22:36:49,989 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524609781_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:36:49,989 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=DF7EAD9946894C5F829999768195E515, index=0
2016-05-17 22:36:49,990 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=DF7EAD9946894C5F829999768195E515, sourceVertex=51745AA8E84E4DE7B80E13C1109C24DC
2016-05-17 22:36:49,990 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=DF7EAD9946894C5F829999768195E515, sourceVertex=073B01178A324857A9F19891CE340F33
2016-05-17 22:36:49,989 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:49,989 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515] : 1
2016-05-17 22:36:49,990 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515] because task failed: task_1463524609781_0001_1_00_000000
2016-05-17 22:36:49,990 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:36:49,990 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:36:49,991 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515]
2016-05-17 22:36:49,991 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=073B01178A324857A9F19891CE340F33, vertexId=vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515]
2016-05-17 22:36:49,989 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:49,990 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524609781_0001_00_000001
2016-05-17 22:36:49,991 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524609781_0001_00_000001
2016-05-17 22:36:49,992 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:49,992 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609781_0001_1][Event:VERTEX_FINISHED]: vertexName=DF7EAD9946894C5F829999768195E515, vertexId=vertex_1463524609781_0001_1_00, initRequestedTime=1463524609937, initedTime=1463524609947, startRequestedTime=1463524609947, startedTime=1463524609947, finishTime=1463524609992, timeTaken=45, status=FAILED, diagnostics=Task failed, taskId=task_1463524609781_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524609957, firstTasksToStart=[ task_1463524609781_0001_1_00_000000 ], lastTaskFinishTime=1463524609988, lastTasksToFinish=[ task_1463524609781_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:36:49,992 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:36:49,992 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609781_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524609781_0001_00_000001, stoppedTime=1463524609992, exitStatus=0
2016-05-17 22:36:49,992 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524609781_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:49,993 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524609781_0001_1_00_000000_0 failed: true container: container_1463524609781_0001_00_000001 numFailedTAs: 0
2016-05-17 22:36:49,993 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:36:49,993 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:36:49,993 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524609781_0001_1
2016-05-17 22:36:49,993 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 073B01178A324857A9F19891CE340F33 for vertex: vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515]
2016-05-17 22:36:49,994 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524609781_0001_1, queueSize=0
2016-05-17 22:36:49,993 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:49,994 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:49,995 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524609781_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524609781_0001_1, startTime=1463524609937, finishTime=1463524609994, timeTaken=57, status=FAILED, diagnostics=Vertex failed, vertexName=DF7EAD9946894C5F829999768195E515, vertexId=vertex_1463524609781_0001_1_00, diagnostics=[Task failed, taskId=task_1463524609781_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524609781_0001_1_00 [DF7EAD9946894C5F829999768195E515] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:36:49,997 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524609781_0001_1 finished with state: FAILED
2016-05-17 22:36:49,993 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524609781_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:36:49,997 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524609781_0001_00_000001 completed but with errors
2016-05-17 22:36:49,997 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524609781_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:49,998 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524609781_0001_00_000001
2016-05-17 22:36:49 Completed Dag: dag_1463524609781_0001_1
2016-05-17 22:36:49,998 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524609781_0001_1, dagState=FAILED
2016-05-17 22:36:49,998 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524609781_0001_1_post
2016-05-17 22:36:49,998 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:49,998 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:49,998 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524609781_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:36:49,999 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524609781_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:36:49,999 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524609781_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:36:50,001 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/ALL_VALUES_SWAP, applicationId=application_1463524609781_0001
2016-05-17 22:36:50,003 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:50,003 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:50,003 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:50,003 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:50,003 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:50,003 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:50,004 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:50,004 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:50,004 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:50,004 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:50,004 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:50,004 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:50,005 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:50,007 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2BB0E167975A4437A042A5F3E565E491/C410CCBB368D40FE9249DBA717248602/.tez/application_1463524609781_0001
2016-05-17 22:36:50,008 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping all jobs
2016-05-17 22:36:50,008 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping: (1/1) ...yplatform/ALL_VALUES_SWAP
2016-05-17 22:36:50,009 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopped all jobs
2016-05-17 22:36:50,009 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_VALUES_SWAP/_temporary
2016-05-17 22:36:50,009 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.245
2016-05-17 22:36:50,009 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_VALUES_SWAP
2016-05-17 22:36:50,030 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:50,030 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:50,057 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.004
2016-05-17 22:36:50,066 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 22:36:50,066 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:50,066 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:50,067 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:50,078 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:50,079 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_ARGS_-1
2016-05-17 22:36:50,086 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:50,086 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:50,086 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_ARGS_-1"]
2016-05-17 22:36:50,086 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:50,086 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:50,086 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:50,087 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...emblyplatform/ALL_ARGS_-1
2016-05-17 22:36:50,087 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:50,098 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:50,102 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:50,103 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:50,103 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:50,111 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C0A08A68799D47A390E4CDBEC6325B66/A5903258858340ECBE9408D93767A51D/.tez/application_1463524610102_0001 doesn't exist and is created
2016-05-17 22:36:50,127 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:50,127 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:50,128 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C0A08A68799D47A390E4CDBEC6325B66/A5903258858340ECBE9408D93767A51D/.tez/application_1463524610102_0001
2016-05-17 22:36:50,130 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524610102_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:50,139 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:50,140 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:50,141 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:50,141 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:50,141 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C0A08A68799D47A390E4CDBEC6325B66/A5903258858340ECBE9408D93767A51D/.tez/application_1463524610102_0001/localmode-log-dir
2016-05-17 22:36:50,141 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C0A08A68799D47A390E4CDBEC6325B66/A5903258858340ECBE9408D93767A51D/.tez/application_1463524610102_0001/localmode-log-dir/history.txt.appattempt_1463524610102_0001_000000, maxErrors=10
2016-05-17 22:36:50,142 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:50,142 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524610102_0001_000000, appSubmitTime=1463524610129, launchTime=1463524610130
2016-05-17 22:36:50,144 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:50,144 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:50,147 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:50,147 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:50,147 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:40070
2016-05-17 22:36:50,148 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:50,150 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524610102_0001_000000, startTime=1463524610150
2016-05-17 22:36:50,150 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:50,150 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:50,227 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:50,227 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:50,227 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...emblyplatform/ALL_ARGS_-1, applicationId=application_1463524610102_0001, dagName=[C0A08A68799D47A390E4CDBEC6325B66/A5903258858340ECBE9408D93767A51D] each/(1/1) ...emblyplatform/ALL_ARGS_-1
2016-05-17 22:36:50,240 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [C0A08A68799D47A390E4CDBEC6325B66/A5903258858340ECBE9408D93767A51D] each/(1/1) ...emblyplatform/ALL_ARGS_-1
2016-05-17 22:36:50,242 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524610102_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C0A08A68799D47A390E4CDBEC6325B66/A5903258858340ECBE9408D93767A51D/.tez/application_1463524610102_0001/localmode-log-dir/dag_1463524610102_0001_1.dot
2016-05-17 22:36:50,242 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C0A08A68799D47A390E4CDBEC6325B66/A5903258858340ECBE9408D93767A51D/.tez/application_1463524610102_0001/localmode-log-dir/dag_1463524610102_0001_1-tez-dag.pb.txt
2016-05-17 22:36:50,246 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524610102_0001_1
2016-05-17 22:36:50,246 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:50,246 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [C0A08A68799D47A390E4CDBEC6325B66/A5903258858340ECBE9408D93767A51D] each/(1/1) ...emblyplatform/ALL_ARGS_-1
2016-05-17 22:36:50 Running Dag: dag_1463524610102_0001_1
2016-05-17 22:36:50,259 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610102_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524610102_0001_1, submitTime=1463524610241
2016-05-17 22:36:50,259 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:50,260 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BC55916478694FE28BE748BF92A7E197
2016-05-17 22:36:50,260 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:50,260 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:50,260 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610102_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524610102_0001_1, initTime=1463524610259
2016-05-17 22:36:50,260 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524610102_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:50,261 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610102_0001_1][Event:DAG_STARTED]: dagID=dag_1463524610102_0001_1, startTime=1463524610261
2016-05-17 22:36:50,261 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:50,261 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524610102_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:50,261 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:50,261 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BC55916478694FE28BE748BF92A7E197 : {A4DF821E73464F65BCDE7B1E4C2DCDA1={InputName=A4DF821E73464F65BCDE7B1E4C2DCDA1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:50,261 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A4DF821E73464F65BCDE7B1E4C2DCDA1, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:50,261 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197]
2016-05-17 22:36:50,262 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197]
2016-05-17 22:36:50,262 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197]
2016-05-17 22:36:50,262 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197]. Starting root input initializers: 1
2016-05-17 22:36:50,262 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:50,262 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...emblyplatform/ALL_ARGS_-1, applicationId=application_1463524610102_0001, dagName=[C0A08A68799D47A390E4CDBEC6325B66/A5903258858340ECBE9408D93767A51D] each/(1/1) ...emblyplatform/ALL_ARGS_-1
2016-05-17 22:36:50,262 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A4DF821E73464F65BCDE7B1E4C2DCDA1 on vertex vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197]
2016-05-17 22:36:50,264 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:50,264 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A4DF821E73464F65BCDE7B1E4C2DCDA1 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:50,264 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:50,265 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:50,266 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:50,266 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A4DF821E73464F65BCDE7B1E4C2DCDA1 on vertex vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197]
2016-05-17 22:36:50,266 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:50,266 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A4DF821E73464F65BCDE7B1E4C2DCDA1=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:50,266 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197] parallelism set to 1
2016-05-17 22:36:50,267 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:50,267 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610102_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524610102_0001_1_00, eventCount=1
2016-05-17 22:36:50,267 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197]
2016-05-17 22:36:50,267 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=8DE38076AC7846CA8FBC3B67EEF88206, vertexId=vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:50,267 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=8DE38076AC7846CA8FBC3B67EEF88206, vertexId=vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197]
2016-05-17 22:36:50,269 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=8DE38076AC7846CA8FBC3B67EEF88206, vertexName=BC55916478694FE28BE748BF92A7E197, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:50,269 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=8DE38076AC7846CA8FBC3B67EEF88206, vertexId=vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197]
2016-05-17 22:36:50,270 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:50,270 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610102_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BC55916478694FE28BE748BF92A7E197, vertexId=vertex_1463524610102_0001_1_00, initRequestedTime=1463524610261, initedTime=1463524610270, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:50,270 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197] with distanceFromRoot: 0
2016-05-17 22:36:50,270 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:50,270 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BC55916478694FE28BE748BF92A7E197
2016-05-17 22:36:50,271 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197]
2016-05-17 22:36:50,271 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:50,271 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610102_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524610102_0001_1_00, startRequestedTime=1463524610270, startedTime=1463524610270
2016-05-17 22:36:50,271 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:50,272 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:50,272 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610102_0001_1][Event:TASK_STARTED]: vertexName=BC55916478694FE28BE748BF92A7E197, taskId=task_1463524610102_0001_1_00_000000, scheduledTime=1463524610272, launchTime=1463524610272
2016-05-17 22:36:50,272 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524610102_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:50,272 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197]
2016-05-17 22:36:50,272 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C0A08A68799D47A390E4CDBEC6325B66/A5903258858340ECBE9408D93767A51D] each/(1/1) ...emblyplatform/ALL_ARGS_-1, VertexName: BC55916478694FE28BE748BF92A7E197, VertexParallelism: 1, TaskAttemptID:attempt_1463524610102_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A4DF821E73464F65BCDE7B1E4C2DCDA1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=8DE38076AC7846CA8FBC3B67EEF88206, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:50,273 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524610102_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:50,273 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:50,274 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:50,274 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:50,274 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:50,274 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524610102_0001_00_000001
2016-05-17 22:36:50,275 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524610102_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:50,276 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524610102_0001_00_000001
2016-05-17 22:36:50,276 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:50,276 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524610102_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:50,277 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:50,277 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610102_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524610102_0001_00_000001, launchTime=1463524610277
2016-05-17 22:36:50,278 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524610102_0001_00_000001
2016-05-17 22:36:50,278 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524610102_0001_1_00_000000_0] to container: [container_1463524610102_0001_00_000001]
2016-05-17 22:36:50,278 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524610102_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:50,279 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524610102_0001_1_00_000000_0] started. Is using containerId: [container_1463524610102_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:50,279 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610102_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BC55916478694FE28BE748BF92A7E197, taskAttemptId=attempt_1463524610102_0001_1_00_000000_0, startTime=1463524610279, containerId=container_1463524610102_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524610102_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:50,279 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524610102_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:50,279 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524610102_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:50,279 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:50,280 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524610102_0001_00_000001 given task: attempt_1463524610102_0001_1_00_000000_0
2016-05-17 22:36:50,280 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524610102_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524610102_0001_1_00_000000_0
2016-05-17 22:36:50 Starting to run new task attempt: attempt_1463524610102_0001_1_00_000000_0
2016-05-17 22:36:50,280 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524610102_0001_1_00_000000_0
2016-05-17 22:36:50,280 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:50,281 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:50,281 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:50,281 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:50,281 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:50,281 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:50,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C0A08A68799D47A390E4CDBEC6325B66/A5903258858340ECBE9408D93767A51D] each/(1/1) ...emblyplatform/ALL_ARGS_-1, VertexName: BC55916478694FE28BE748BF92A7E197, VertexParallelism: 1, TaskAttemptID:attempt_1463524610102_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A4DF821E73464F65BCDE7B1E4C2DCDA1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=8DE38076AC7846CA8FBC3B67EEF88206, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:50,281 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524610102_0001, with dag id: dag_1463524610102_0001_1
2016-05-17 22:36:50,281 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:50,282 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524610102_0001_1_00_000000_0
2016-05-17 22:36:50,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:50,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:50,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A4DF821E73464F65BCDE7B1E4C2DCDA1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:50,283 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:50,283 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A4DF821E73464F65BCDE7B1E4C2DCDA1
2016-05-17 22:36:50,283 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A4DF821E73464F65BCDE7B1E4C2DCDA1
2016-05-17 22:36:50,284 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524610102_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:50,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:50,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=8DE38076AC7846CA8FBC3B67EEF88206, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:50,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:50,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 8DE38076AC7846CA8FBC3B67EEF88206
2016-05-17 22:36:50,286 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:50,286 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 8DE38076AC7846CA8FBC3B67EEF88206
2016-05-17 22:36:50,288 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:50,290 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:50,290 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A4DF821E73464F65BCDE7B1E4C2DCDA1
2016-05-17 22:36:50,290 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A4DF821E73464F65BCDE7B1E4C2DCDA1
2016-05-17 22:36:50,293 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BC55916478694FE28BE748BF92A7E197, ordinal: 0
2016-05-17 22:36:50,293 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BC55916478694FE28BE748BF92A7E197, mem on start (mb), free: 125, total: 305, max: 672
2016-05-17 22:36:50,294 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:50,294 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:50,294 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:50,296 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:50,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 8DE38076AC7846CA8FBC3B67EEF88206
2016-05-17 22:36:50,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:50,296 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:50,296 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:50,296 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:50,296 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:50,296 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:50,296 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:50,296 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:50,297 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A4DF821E73464F65BCDE7B1E4C2DCDA1, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:50,297 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 8DE38076AC7846CA8FBC3B67EEF88206, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:50,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:50,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A4DF821E73464F65BCDE7B1E4C2DCDA1 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:50,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A4DF821E73464F65BCDE7B1E4C2DCDA1
2016-05-17 22:36:50,297 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:50,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A4DF821E73464F65BCDE7B1E4C2DCDA1
2016-05-17 22:36:50,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:50,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:50,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:50,297 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524610102_0001_1_00_000000_0
2016-05-17 22:36:50,298 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:50,299 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4b5b347d
2016-05-17 22:36:50,299 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4b5b347d
2016-05-17 22:36:50,299 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:50,300 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:50,300 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:50,300 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: A4DF821E73464F65BCDE7B1E4C2DCDA1
2016-05-17 22:36:50,301 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_ARGS_-1"], id: 8DE38076AC7846CA8FBC3B67EEF88206
2016-05-17 22:36:50,301 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_ARGS_-1"]
2016-05-17 22:36:50,301 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:50,301 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:50,301 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BC55916478694FE28BE748BF92A7E197, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:50,302 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:50,302 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:50,302 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:50,302 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:50,303 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524610102_0001_1_00_000000_0
2016-05-17 22:36:50,303 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524610102_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:50,303 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524610102_0001_1_00_000000_0
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:50,304 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=319815680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524610301, Process_Duration=2, Process_End_Time=1463524610303, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:36:50,304 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BC55916478694FE28BE748BF92A7E197, mem on close (mb), free: 123, total: 305, max: 672
2016-05-17 22:36:50,304 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=BC55916478694FE28BE748BF92A7E197, index=0
2016-05-17 22:36:50,305 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=BC55916478694FE28BE748BF92A7E197, sourceVertex=A4DF821E73464F65BCDE7B1E4C2DCDA1
2016-05-17 22:36:50,305 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610102_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BC55916478694FE28BE748BF92A7E197, taskAttemptId=attempt_1463524610102_0001_1_00_000000_0, startTime=1463524610279, finishTime=1463524610304, timeTaken=25, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=319815680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524610301, Process_Duration=2, Process_End_Time=1463524610303, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:36:50,305 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524610102_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:36:50,305 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524610102_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:36:50,305 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:50,305 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610102_0001_1][Event:TASK_FINISHED]: vertexName=BC55916478694FE28BE748BF92A7E197, taskId=task_1463524610102_0001_1_00_000000, startTime=1463524610279, finishTime=1463524610305, timeTaken=26, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
], counters=Counters: 0
2016-05-17 22:36:50,305 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524610102_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:36:50,306 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:50,306 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=BC55916478694FE28BE748BF92A7E197, sourceVertex=8DE38076AC7846CA8FBC3B67EEF88206
2016-05-17 22:36:50,306 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197] : 1
2016-05-17 22:36:50,306 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197] because task failed: task_1463524610102_0001_1_00_000000
2016-05-17 22:36:50,306 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:36:50,306 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:36:50,306 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197]
2016-05-17 22:36:50,306 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=8DE38076AC7846CA8FBC3B67EEF88206, vertexId=vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197]
2016-05-17 22:36:50,307 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610102_0001_1][Event:VERTEX_FINISHED]: vertexName=BC55916478694FE28BE748BF92A7E197, vertexId=vertex_1463524610102_0001_1_00, initRequestedTime=1463524610261, initedTime=1463524610270, startRequestedTime=1463524610270, startedTime=1463524610270, finishTime=1463524610307, timeTaken=37, status=FAILED, diagnostics=Task failed, taskId=task_1463524610102_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524610279, firstTasksToStart=[ task_1463524610102_0001_1_00_000000 ], lastTaskFinishTime=1463524610304, lastTasksToFinish=[ task_1463524610102_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:36:50,308 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:36:50,308 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610102_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524610102_0001_00_000001, stoppedTime=1463524610308, exitStatus=0
2016-05-17 22:36:50,308 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524610102_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:50,308 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524610102_0001_1_00_000000_0 failed: true container: container_1463524610102_0001_00_000001 numFailedTAs: 0
2016-05-17 22:36:50,308 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:36:50,308 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:36:50,308 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524610102_0001_1
2016-05-17 22:36:50,308 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 8DE38076AC7846CA8FBC3B67EEF88206 for vertex: vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197]
2016-05-17 22:36:50,306 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:50,309 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:50,309 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524610102_0001_1, queueSize=0
2016-05-17 22:36:50,309 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610102_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524610102_0001_1, startTime=1463524610261, finishTime=1463524610309, timeTaken=48, status=FAILED, diagnostics=Vertex failed, vertexName=BC55916478694FE28BE748BF92A7E197, vertexId=vertex_1463524610102_0001_1_00, diagnostics=[Task failed, taskId=task_1463524610102_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524610102_0001_1_00 [BC55916478694FE28BE748BF92A7E197] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:36:50,309 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:50,309 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524610102_0001_1 finished with state: FAILED
2016-05-17 22:36:50,310 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:50,310 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524610102_0001_00_000001
2016-05-17 22:36:50,310 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524610102_0001_00_000001
2016-05-17 22:36:50,310 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524610102_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:50 Completed Dag: dag_1463524610102_0001_1
2016-05-17 22:36:50,310 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524610102_0001_1, dagState=FAILED
2016-05-17 22:36:50,310 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524610102_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:36:50,310 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524610102_0001_1_post
2016-05-17 22:36:50,311 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:50,311 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:50,311 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524610102_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:36:50,311 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524610102_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:36:50,312 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524610102_0001_00_000001 completed but with errors
2016-05-17 22:36:50,312 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524610102_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:36:50,312 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524610102_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:36:50,312 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524610102_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:36:50,314 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...emblyplatform/ALL_ARGS_-1, applicationId=application_1463524610102_0001
2016-05-17 22:36:50,315 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:50,316 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:50,316 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:50,316 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:50,316 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:50,316 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:50,317 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:50,317 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:50,317 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:50,317 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:50,318 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:50,318 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:50,318 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:50,321 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C0A08A68799D47A390E4CDBEC6325B66/A5903258858340ECBE9408D93767A51D/.tez/application_1463524610102_0001
2016-05-17 22:36:50,321 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping all jobs
2016-05-17 22:36:50,322 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping: (1/1) ...emblyplatform/ALL_ARGS_-1
2016-05-17 22:36:50,322 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopped all jobs
2016-05-17 22:36:50,322 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_ARGS_-1/_temporary
2016-05-17 22:36:50,323 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.236
2016-05-17 22:36:50,323 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_ARGS_-1
2016-05-17 22:36:50,342 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:50,342 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:50,366 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.001
2016-05-17 22:36:50,368 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-05-17 22:36:50,369 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:50,369 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:50,369 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_ARGS_'field'
2016-05-17 22:36:50,374 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:50,374 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:50,392 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:50,392 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:50,418 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-05-17 22:36:50,421 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-05-17 22:36:50,421 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:50,421 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:50,421 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:50,433 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:50,434 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_ARGS_RESULTS
2016-05-17 22:36:50,441 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:50,441 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:50,442 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_ARGS_RESULTS"]
2016-05-17 22:36:50,442 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:50,442 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:50,442 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:50,442 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...platform/ALL_ARGS_RESULTS
2016-05-17 22:36:50,443 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:50,463 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:50,467 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:50,467 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:50,468 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:50,477 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/741457EF5B8C40C2BE02EF851E9CF217/E2D75101D1EE490EAE2575686CDC1A0E/.tez/application_1463524610467_0001 doesn't exist and is created
2016-05-17 22:36:50,492 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:50,492 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:50,493 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/741457EF5B8C40C2BE02EF851E9CF217/E2D75101D1EE490EAE2575686CDC1A0E/.tez/application_1463524610467_0001
2016-05-17 22:36:50,495 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524610467_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:50,506 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:50,507 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:50,508 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:50,508 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:50,508 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/741457EF5B8C40C2BE02EF851E9CF217/E2D75101D1EE490EAE2575686CDC1A0E/.tez/application_1463524610467_0001/localmode-log-dir
2016-05-17 22:36:50,508 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/741457EF5B8C40C2BE02EF851E9CF217/E2D75101D1EE490EAE2575686CDC1A0E/.tez/application_1463524610467_0001/localmode-log-dir/history.txt.appattempt_1463524610467_0001_000000, maxErrors=10
2016-05-17 22:36:50,508 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:50,508 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524610467_0001_000000, appSubmitTime=1463524610495, launchTime=1463524610495
2016-05-17 22:36:50,511 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:50,511 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:50,512 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:50,513 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:50,513 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:54938
2016-05-17 22:36:50,514 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:50,516 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524610467_0001_000000, startTime=1463524610516
2016-05-17 22:36:50,516 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:50,516 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:50,593 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:50,593 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:50,593 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/ALL_ARGS_RESULTS, applicationId=application_1463524610467_0001, dagName=[741457EF5B8C40C2BE02EF851E9CF217/E2D75101D1EE490EAE2575686CDC1A0E] each/(1/1) ...platform/ALL_ARGS_RESULTS
2016-05-17 22:36:50,606 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [741457EF5B8C40C2BE02EF851E9CF217/E2D75101D1EE490EAE2575686CDC1A0E] each/(1/1) ...platform/ALL_ARGS_RESULTS
2016-05-17 22:36:50,607 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524610467_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/741457EF5B8C40C2BE02EF851E9CF217/E2D75101D1EE490EAE2575686CDC1A0E/.tez/application_1463524610467_0001/localmode-log-dir/dag_1463524610467_0001_1.dot
2016-05-17 22:36:50,608 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/741457EF5B8C40C2BE02EF851E9CF217/E2D75101D1EE490EAE2575686CDC1A0E/.tez/application_1463524610467_0001/localmode-log-dir/dag_1463524610467_0001_1-tez-dag.pb.txt
2016-05-17 22:36:50,612 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524610467_0001_1
2016-05-17 22:36:50,612 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:50,612 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [741457EF5B8C40C2BE02EF851E9CF217/E2D75101D1EE490EAE2575686CDC1A0E] each/(1/1) ...platform/ALL_ARGS_RESULTS
2016-05-17 22:36:50 Running Dag: dag_1463524610467_0001_1
2016-05-17 22:36:50,622 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:50,623 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610467_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524610467_0001_1, submitTime=1463524610606
2016-05-17 22:36:50,623 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DD28ADD9E9DF4ED5802CA76E25F2E3D8
2016-05-17 22:36:50,623 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:50,623 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:50,624 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610467_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524610467_0001_1, initTime=1463524610623
2016-05-17 22:36:50,624 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524610467_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:50,624 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610467_0001_1][Event:DAG_STARTED]: dagID=dag_1463524610467_0001_1, startTime=1463524610624
2016-05-17 22:36:50,624 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:50,624 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524610467_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:50,624 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:50,624 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: DD28ADD9E9DF4ED5802CA76E25F2E3D8 : {CEB63F7DD3794B5ABFC69F6F551A4535={InputName=CEB63F7DD3794B5ABFC69F6F551A4535}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:50,624 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: CEB63F7DD3794B5ABFC69F6F551A4535, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:50,624 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8]
2016-05-17 22:36:50,625 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8]
2016-05-17 22:36:50,625 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8]
2016-05-17 22:36:50,625 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8]. Starting root input initializers: 1
2016-05-17 22:36:50,625 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/ALL_ARGS_RESULTS, applicationId=application_1463524610467_0001, dagName=[741457EF5B8C40C2BE02EF851E9CF217/E2D75101D1EE490EAE2575686CDC1A0E] each/(1/1) ...platform/ALL_ARGS_RESULTS
2016-05-17 22:36:50,625 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:50,625 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: CEB63F7DD3794B5ABFC69F6F551A4535 on vertex vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8]
2016-05-17 22:36:50,627 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:50,627 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input CEB63F7DD3794B5ABFC69F6F551A4535 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:50,628 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:50,628 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:50,629 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:50,629 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: CEB63F7DD3794B5ABFC69F6F551A4535 on vertex vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8]
2016-05-17 22:36:50,630 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:50,630 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {CEB63F7DD3794B5ABFC69F6F551A4535=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:50,630 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8] parallelism set to 1
2016-05-17 22:36:50,630 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:50,630 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610467_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524610467_0001_1_00, eventCount=1
2016-05-17 22:36:50,630 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8]
2016-05-17 22:36:50,630 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=B831F6C516D34699A1C4BF1C5A0B161B, vertexId=vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:50,630 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=B831F6C516D34699A1C4BF1C5A0B161B, vertexId=vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8]
2016-05-17 22:36:50,632 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=B831F6C516D34699A1C4BF1C5A0B161B, vertexName=DD28ADD9E9DF4ED5802CA76E25F2E3D8, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:50,632 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=B831F6C516D34699A1C4BF1C5A0B161B, vertexId=vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8]
2016-05-17 22:36:50,633 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610467_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DD28ADD9E9DF4ED5802CA76E25F2E3D8, vertexId=vertex_1463524610467_0001_1_00, initRequestedTime=1463524610624, initedTime=1463524610633, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:50,633 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8] with distanceFromRoot: 0
2016-05-17 22:36:50,633 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:50,634 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:50,634 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in DD28ADD9E9DF4ED5802CA76E25F2E3D8
2016-05-17 22:36:50,634 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8]
2016-05-17 22:36:50,634 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:50,634 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610467_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524610467_0001_1_00, startRequestedTime=1463524610634, startedTime=1463524610634
2016-05-17 22:36:50,635 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:50,637 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:50,637 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610467_0001_1][Event:TASK_STARTED]: vertexName=DD28ADD9E9DF4ED5802CA76E25F2E3D8, taskId=task_1463524610467_0001_1_00_000000, scheduledTime=1463524610636, launchTime=1463524610636
2016-05-17 22:36:50,637 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524610467_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:50,638 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8]
2016-05-17 22:36:50,638 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [741457EF5B8C40C2BE02EF851E9CF217/E2D75101D1EE490EAE2575686CDC1A0E] each/(1/1) ...platform/ALL_ARGS_RESULTS, VertexName: DD28ADD9E9DF4ED5802CA76E25F2E3D8, VertexParallelism: 1, TaskAttemptID:attempt_1463524610467_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CEB63F7DD3794B5ABFC69F6F551A4535, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B831F6C516D34699A1C4BF1C5A0B161B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:50,638 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524610467_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:50,638 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:50,638 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524610467_0001, with dag id: dag_1463524610467_0001_1
2016-05-17 22:36:50,639 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:50,640 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:50,640 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:50,640 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524610467_0001_00_000001
2016-05-17 22:36:50,641 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524610467_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:50,643 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524610467_0001_00_000001
2016-05-17 22:36:50,643 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:50,643 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524610467_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:50,644 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:50,644 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610467_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524610467_0001_00_000001, launchTime=1463524610643
2016-05-17 22:36:50,644 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524610467_0001_00_000001
2016-05-17 22:36:50,644 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524610467_0001_1_00_000000_0] to container: [container_1463524610467_0001_00_000001]
2016-05-17 22:36:50,644 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524610467_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:50,645 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524610467_0001_00_000001 given task: attempt_1463524610467_0001_1_00_000000_0
2016-05-17 22:36:50,645 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524610467_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524610467_0001_1_00_000000_0
2016-05-17 22:36:50,645 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524610467_0001_1_00_000000_0] started. Is using containerId: [container_1463524610467_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:50,645 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610467_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DD28ADD9E9DF4ED5802CA76E25F2E3D8, taskAttemptId=attempt_1463524610467_0001_1_00_000000_0, startTime=1463524610645, containerId=container_1463524610467_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524610467_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:50,645 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524610467_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:50,645 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524610467_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:50,645 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:50 Starting to run new task attempt: attempt_1463524610467_0001_1_00_000000_0
2016-05-17 22:36:50,646 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524610467_0001_1_00_000000_0
2016-05-17 22:36:50,646 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:50,646 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:50,646 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:50,646 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:50,646 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:50,646 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:50,646 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [741457EF5B8C40C2BE02EF851E9CF217/E2D75101D1EE490EAE2575686CDC1A0E] each/(1/1) ...platform/ALL_ARGS_RESULTS, VertexName: DD28ADD9E9DF4ED5802CA76E25F2E3D8, VertexParallelism: 1, TaskAttemptID:attempt_1463524610467_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CEB63F7DD3794B5ABFC69F6F551A4535, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B831F6C516D34699A1C4BF1C5A0B161B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:50,646 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:50,646 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524610467_0001_1_00_000000_0
2016-05-17 22:36:50,646 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:50,646 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:50,647 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CEB63F7DD3794B5ABFC69F6F551A4535, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:50,647 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:50,647 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:50,647 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524610467_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:50,647 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CEB63F7DD3794B5ABFC69F6F551A4535
2016-05-17 22:36:50,648 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: CEB63F7DD3794B5ABFC69F6F551A4535
2016-05-17 22:36:50,649 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B831F6C516D34699A1C4BF1C5A0B161B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:50,649 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:50,650 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: CEB63F7DD3794B5ABFC69F6F551A4535
2016-05-17 22:36:50,650 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CEB63F7DD3794B5ABFC69F6F551A4535
2016-05-17 22:36:50,650 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:50,650 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:50,651 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B831F6C516D34699A1C4BF1C5A0B161B
2016-05-17 22:36:50,651 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:50,651 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: B831F6C516D34699A1C4BF1C5A0B161B
2016-05-17 22:36:50,656 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DD28ADD9E9DF4ED5802CA76E25F2E3D8, ordinal: 0
2016-05-17 22:36:50,656 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD28ADD9E9DF4ED5802CA76E25F2E3D8, mem on start (mb), free: 199, total: 314, max: 672
2016-05-17 22:36:50,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:50,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:50,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:50,660 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:50,660 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B831F6C516D34699A1C4BF1C5A0B161B
2016-05-17 22:36:50,660 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:50,660 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:50,660 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:50,660 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:50,661 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:50,661 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:50,661 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:50,661 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:50,661 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CEB63F7DD3794B5ABFC69F6F551A4535, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:50,661 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B831F6C516D34699A1C4BF1C5A0B161B, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:50,662 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:50,662 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CEB63F7DD3794B5ABFC69F6F551A4535 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:50,662 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:50,662 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CEB63F7DD3794B5ABFC69F6F551A4535
2016-05-17 22:36:50,662 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:50,662 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CEB63F7DD3794B5ABFC69F6F551A4535
2016-05-17 22:36:50,662 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:50,662 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:50,663 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524610467_0001_1_00_000000_0
2016-05-17 22:36:50,663 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:50,663 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4c17083a
2016-05-17 22:36:50,664 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4c17083a
2016-05-17 22:36:50,664 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:50,664 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:50,667 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:50,667 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: CEB63F7DD3794B5ABFC69F6F551A4535
2016-05-17 22:36:50,668 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_ARGS_RESULTS"], id: B831F6C516D34699A1C4BF1C5A0B161B
2016-05-17 22:36:50,668 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_ARGS_RESULTS"]
2016-05-17 22:36:50,668 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:50,668 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:50,669 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DD28ADD9E9DF4ED5802CA76E25F2E3D8, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:50,669 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:50,673 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:50,676 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:50,677 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:50,678 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524610467_0001_1_00_000000_0
2016-05-17 22:36:50,678 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524610467_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:50,679 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524610467_0001_1_00_000000_0
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:50,683 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=329777152, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524610669, Process_Duration=10, Process_End_Time=1463524610679, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:36:50,683 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610467_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DD28ADD9E9DF4ED5802CA76E25F2E3D8, taskAttemptId=attempt_1463524610467_0001_1_00_000000_0, startTime=1463524610645, finishTime=1463524610683, timeTaken=38, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=329777152, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524610669, Process_Duration=10, Process_End_Time=1463524610679, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:36:50,683 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524610467_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:36:50,683 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524610467_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:36:50,684 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610467_0001_1][Event:TASK_FINISHED]: vertexName=DD28ADD9E9DF4ED5802CA76E25F2E3D8, taskId=task_1463524610467_0001_1_00_000000, startTime=1463524610645, finishTime=1463524610683, timeTaken=38, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
], counters=Counters: 0
2016-05-17 22:36:50,684 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524610467_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:36:50,684 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:50,684 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:50,685 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:50,683 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD28ADD9E9DF4ED5802CA76E25F2E3D8, mem on close (mb), free: 197, total: 314, max: 672
2016-05-17 22:36:50,685 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=DD28ADD9E9DF4ED5802CA76E25F2E3D8, index=0
2016-05-17 22:36:50,684 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8] : 1
2016-05-17 22:36:50,685 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8] because task failed: task_1463524610467_0001_1_00_000000
2016-05-17 22:36:50,686 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:36:50,685 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=DD28ADD9E9DF4ED5802CA76E25F2E3D8, sourceVertex=CEB63F7DD3794B5ABFC69F6F551A4535
2016-05-17 22:36:50,686 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:36:50,686 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=DD28ADD9E9DF4ED5802CA76E25F2E3D8, sourceVertex=B831F6C516D34699A1C4BF1C5A0B161B
2016-05-17 22:36:50,686 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8]
2016-05-17 22:36:50,686 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=B831F6C516D34699A1C4BF1C5A0B161B, vertexId=vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8]
2016-05-17 22:36:50,687 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:50,687 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610467_0001_1][Event:VERTEX_FINISHED]: vertexName=DD28ADD9E9DF4ED5802CA76E25F2E3D8, vertexId=vertex_1463524610467_0001_1_00, initRequestedTime=1463524610624, initedTime=1463524610633, startRequestedTime=1463524610634, startedTime=1463524610634, finishTime=1463524610687, timeTaken=53, status=FAILED, diagnostics=Task failed, taskId=task_1463524610467_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524610645, firstTasksToStart=[ task_1463524610467_0001_1_00_000000 ], lastTaskFinishTime=1463524610683, lastTasksToFinish=[ task_1463524610467_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:36:50,687 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:36:50,687 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610467_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524610467_0001_00_000001, stoppedTime=1463524610687, exitStatus=0
2016-05-17 22:36:50,687 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524610467_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:50,687 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:50,688 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524610467_0001_1_00_000000_0 failed: true container: container_1463524610467_0001_00_000001 numFailedTAs: 0
2016-05-17 22:36:50,688 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:36:50,688 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:36:50,688 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524610467_0001_1
2016-05-17 22:36:50,688 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: B831F6C516D34699A1C4BF1C5A0B161B for vertex: vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8]
2016-05-17 22:36:50,688 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524610467_0001_00_000001
2016-05-17 22:36:50,688 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524610467_0001_00_000001
2016-05-17 22:36:50,689 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524610467_0001_1, queueSize=0
2016-05-17 22:36:50,689 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:50,689 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610467_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524610467_0001_1, startTime=1463524610624, finishTime=1463524610689, timeTaken=65, status=FAILED, diagnostics=Vertex failed, vertexName=DD28ADD9E9DF4ED5802CA76E25F2E3D8, vertexId=vertex_1463524610467_0001_1_00, diagnostics=[Task failed, taskId=task_1463524610467_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524610467_0001_1_00 [DD28ADD9E9DF4ED5802CA76E25F2E3D8] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:36:50,690 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524610467_0001_1 finished with state: FAILED
2016-05-17 22:36:50,690 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524610467_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:50,690 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524610467_0001_00_000001
2016-05-17 22:36:50 Completed Dag: dag_1463524610467_0001_1
2016-05-17 22:36:50,690 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524610467_0001_1, dagState=FAILED
2016-05-17 22:36:50,690 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524610467_0001_1_post
2016-05-17 22:36:50,690 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:50,690 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:50,690 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524610467_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:36:50,691 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524610467_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:36:50,692 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524610467_0001_00_000001 completed but with errors
2016-05-17 22:36:50,693 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524610467_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:36:50,694 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524610467_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:36:50,694 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524610467_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:36:50,695 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/ALL_ARGS_RESULTS, applicationId=application_1463524610467_0001
2016-05-17 22:36:50,696 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:50,697 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:50,697 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:50,697 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:50,697 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:50,697 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:50,697 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:50,697 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:50,698 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:50,698 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:50,698 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:50,698 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:50,698 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:50,701 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/741457EF5B8C40C2BE02EF851E9CF217/E2D75101D1EE490EAE2575686CDC1A0E/.tez/application_1463524610467_0001
2016-05-17 22:36:50,702 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping all jobs
2016-05-17 22:36:50,702 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping: (1/1) ...platform/ALL_ARGS_RESULTS
2016-05-17 22:36:50,703 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopped all jobs
2016-05-17 22:36:50,703 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_ARGS_RESULTS/_temporary
2016-05-17 22:36:50,703 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.262
2016-05-17 22:36:50,703 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_ARGS_RESULTS
2016-05-17 22:36:50,733 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:50,733 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:50,759 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.001
2016-05-17 22:36:50,761 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-05-17 22:36:50,761 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:50,762 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:36:50,762 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_ARGS_ALL
2016-05-17 22:36:50,769 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:50,769 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:50,780 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:50,781 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:50,806 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-05-17 22:36:50,809 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-05-17 22:36:50,810 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:50,810 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:50,810 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:50,825 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:50,825 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_ARGS_REPLACE
2016-05-17 22:36:50,832 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:50,832 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:50,832 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_ARGS_REPLACE"]
2016-05-17 22:36:50,832 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:50,832 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:50,833 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:50,834 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...platform/ALL_ARGS_REPLACE
2016-05-17 22:36:50,834 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:50,845 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:50,849 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:50,850 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:50,850 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:50,860 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5DA426B46D4645298F74335B2AA9A24D/DD27B7B50F0D4C209BAC2BC818D683B9/.tez/application_1463524610850_0001 doesn't exist and is created
2016-05-17 22:36:50,875 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:50,875 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:50,876 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5DA426B46D4645298F74335B2AA9A24D/DD27B7B50F0D4C209BAC2BC818D683B9/.tez/application_1463524610850_0001
2016-05-17 22:36:50,877 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524610850_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:50,887 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:50,887 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:50,888 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:50,888 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:50,888 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5DA426B46D4645298F74335B2AA9A24D/DD27B7B50F0D4C209BAC2BC818D683B9/.tez/application_1463524610850_0001/localmode-log-dir
2016-05-17 22:36:50,889 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5DA426B46D4645298F74335B2AA9A24D/DD27B7B50F0D4C209BAC2BC818D683B9/.tez/application_1463524610850_0001/localmode-log-dir/history.txt.appattempt_1463524610850_0001_000000, maxErrors=10
2016-05-17 22:36:50,889 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:50,889 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524610850_0001_000000, appSubmitTime=1463524610877, launchTime=1463524610877
2016-05-17 22:36:50,893 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:50,894 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:34007
2016-05-17 22:36:50,894 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:50,895 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:50,895 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:50,896 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:50,898 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524610850_0001_000000, startTime=1463524610898
2016-05-17 22:36:50,898 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:50,898 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:50,975 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:50,975 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:50,975 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/ALL_ARGS_REPLACE, applicationId=application_1463524610850_0001, dagName=[5DA426B46D4645298F74335B2AA9A24D/DD27B7B50F0D4C209BAC2BC818D683B9] each/(1/1) ...platform/ALL_ARGS_REPLACE
2016-05-17 22:36:50,989 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [5DA426B46D4645298F74335B2AA9A24D/DD27B7B50F0D4C209BAC2BC818D683B9] each/(1/1) ...platform/ALL_ARGS_REPLACE
2016-05-17 22:36:50,991 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524610850_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5DA426B46D4645298F74335B2AA9A24D/DD27B7B50F0D4C209BAC2BC818D683B9/.tez/application_1463524610850_0001/localmode-log-dir/dag_1463524610850_0001_1.dot
2016-05-17 22:36:50,991 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5DA426B46D4645298F74335B2AA9A24D/DD27B7B50F0D4C209BAC2BC818D683B9/.tez/application_1463524610850_0001/localmode-log-dir/dag_1463524610850_0001_1-tez-dag.pb.txt
2016-05-17 22:36:50,995 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524610850_0001_1
2016-05-17 22:36:50,995 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:50,995 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [5DA426B46D4645298F74335B2AA9A24D/DD27B7B50F0D4C209BAC2BC818D683B9] each/(1/1) ...platform/ALL_ARGS_REPLACE
2016-05-17 22:36:50 Running Dag: dag_1463524610850_0001_1
2016-05-17 22:36:51,007 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610850_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524610850_0001_1, submitTime=1463524610989
2016-05-17 22:36:51,007 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:51,007 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D3E3A626697C4596980B621EAE6C24AB
2016-05-17 22:36:51,008 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:51,008 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610850_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524610850_0001_1, initTime=1463524611007
2016-05-17 22:36:51,008 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:51,008 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524610850_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:51,008 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610850_0001_1][Event:DAG_STARTED]: dagID=dag_1463524610850_0001_1, startTime=1463524611008
2016-05-17 22:36:51,008 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:51,008 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524610850_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:51,008 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:51,009 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D3E3A626697C4596980B621EAE6C24AB : {4B21243D5AAA45058FADED9495728FC9={InputName=4B21243D5AAA45058FADED9495728FC9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:51,009 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 4B21243D5AAA45058FADED9495728FC9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:51,009 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB]
2016-05-17 22:36:51,009 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB]
2016-05-17 22:36:51,009 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB]
2016-05-17 22:36:51,009 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB]. Starting root input initializers: 1
2016-05-17 22:36:51,009 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:51,010 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 4B21243D5AAA45058FADED9495728FC9 on vertex vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB]
2016-05-17 22:36:51,010 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/ALL_ARGS_REPLACE, applicationId=application_1463524610850_0001, dagName=[5DA426B46D4645298F74335B2AA9A24D/DD27B7B50F0D4C209BAC2BC818D683B9] each/(1/1) ...platform/ALL_ARGS_REPLACE
2016-05-17 22:36:51,012 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:51,012 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 4B21243D5AAA45058FADED9495728FC9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:51,013 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:51,013 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:51,014 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:51,014 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 4B21243D5AAA45058FADED9495728FC9 on vertex vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB]
2016-05-17 22:36:51,015 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:51,015 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {4B21243D5AAA45058FADED9495728FC9=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:51,015 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB] parallelism set to 1
2016-05-17 22:36:51,015 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:51,015 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610850_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524610850_0001_1_00, eventCount=1
2016-05-17 22:36:51,015 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB]
2016-05-17 22:36:51,015 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=15D6E6686FC74758872A3763D6673E6B, vertexId=vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:51,015 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=15D6E6686FC74758872A3763D6673E6B, vertexId=vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB]
2016-05-17 22:36:51,017 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=15D6E6686FC74758872A3763D6673E6B, vertexName=D3E3A626697C4596980B621EAE6C24AB, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:51,017 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=15D6E6686FC74758872A3763D6673E6B, vertexId=vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB]
2016-05-17 22:36:51,018 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:51,019 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610850_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D3E3A626697C4596980B621EAE6C24AB, vertexId=vertex_1463524610850_0001_1_00, initRequestedTime=1463524611009, initedTime=1463524611018, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:51,019 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB] with distanceFromRoot: 0
2016-05-17 22:36:51,019 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:51,019 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D3E3A626697C4596980B621EAE6C24AB
2016-05-17 22:36:51,019 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB]
2016-05-17 22:36:51,020 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:51,020 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610850_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524610850_0001_1_00, startRequestedTime=1463524611019, startedTime=1463524611019
2016-05-17 22:36:51,020 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:51,023 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:51,023 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610850_0001_1][Event:TASK_STARTED]: vertexName=D3E3A626697C4596980B621EAE6C24AB, taskId=task_1463524610850_0001_1_00_000000, scheduledTime=1463524611023, launchTime=1463524611023
2016-05-17 22:36:51,023 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524610850_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:51,023 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB]
2016-05-17 22:36:51,023 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5DA426B46D4645298F74335B2AA9A24D/DD27B7B50F0D4C209BAC2BC818D683B9] each/(1/1) ...platform/ALL_ARGS_REPLACE, VertexName: D3E3A626697C4596980B621EAE6C24AB, VertexParallelism: 1, TaskAttemptID:attempt_1463524610850_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4B21243D5AAA45058FADED9495728FC9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=15D6E6686FC74758872A3763D6673E6B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:51,023 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524610850_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:51,023 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:51,024 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:51,024 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:51,024 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:51,024 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524610850_0001_00_000001
2016-05-17 22:36:51,025 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524610850_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:51,026 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524610850_0001_00_000001
2016-05-17 22:36:51,026 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:51,026 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:51,026 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610850_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524610850_0001_00_000001, launchTime=1463524611026
2016-05-17 22:36:51,026 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524610850_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:51,027 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524610850_0001_00_000001
2016-05-17 22:36:51,027 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524610850_0001_1_00_000000_0] to container: [container_1463524610850_0001_00_000001]
2016-05-17 22:36:51,027 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524610850_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:51,027 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524610850_0001_00_000001 given task: attempt_1463524610850_0001_1_00_000000_0
2016-05-17 22:36:51,027 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524610850_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524610850_0001_1_00_000000_0
2016-05-17 22:36:51,028 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524610850_0001_1_00_000000_0] started. Is using containerId: [container_1463524610850_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:51 Starting to run new task attempt: attempt_1463524610850_0001_1_00_000000_0
2016-05-17 22:36:51,028 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524610850_0001_1_00_000000_0
2016-05-17 22:36:51,028 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:51,028 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:51,028 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:51,028 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:51,028 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:51,028 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:51,028 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:51,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5DA426B46D4645298F74335B2AA9A24D/DD27B7B50F0D4C209BAC2BC818D683B9] each/(1/1) ...platform/ALL_ARGS_REPLACE, VertexName: D3E3A626697C4596980B621EAE6C24AB, VertexParallelism: 1, TaskAttemptID:attempt_1463524610850_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4B21243D5AAA45058FADED9495728FC9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=15D6E6686FC74758872A3763D6673E6B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:51,029 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610850_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D3E3A626697C4596980B621EAE6C24AB, taskAttemptId=attempt_1463524610850_0001_1_00_000000_0, startTime=1463524611027, containerId=container_1463524610850_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524610850_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:51,029 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:51,029 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524610850_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:51,029 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524610850_0001_1_00_000000_0
2016-05-17 22:36:51,029 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:51,029 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:51,029 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524610850_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:51,030 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=4B21243D5AAA45058FADED9495728FC9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:51,030 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:51,031 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:51,031 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 4B21243D5AAA45058FADED9495728FC9
2016-05-17 22:36:51,031 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 4B21243D5AAA45058FADED9495728FC9
2016-05-17 22:36:51,033 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:51,033 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:51,033 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 4B21243D5AAA45058FADED9495728FC9
2016-05-17 22:36:51,034 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 4B21243D5AAA45058FADED9495728FC9
2016-05-17 22:36:51,030 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524610850_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:51,034 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524610850_0001, with dag id: dag_1463524610850_0001_1
2016-05-17 22:36:51,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=15D6E6686FC74758872A3763D6673E6B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:51,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:51,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 15D6E6686FC74758872A3763D6673E6B
2016-05-17 22:36:51,036 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:51,037 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 15D6E6686FC74758872A3763D6673E6B
2016-05-17 22:36:51,041 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D3E3A626697C4596980B621EAE6C24AB, ordinal: 0
2016-05-17 22:36:51,041 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D3E3A626697C4596980B621EAE6C24AB, mem on start (mb), free: 122, total: 314, max: 672
2016-05-17 22:36:51,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:51,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:51,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:51,048 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:51,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 15D6E6686FC74758872A3763D6673E6B
2016-05-17 22:36:51,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:51,048 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:51,048 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:51,048 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:51,048 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:51,048 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:51,048 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:51,049 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:51,049 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 4B21243D5AAA45058FADED9495728FC9, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:51,049 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 15D6E6686FC74758872A3763D6673E6B, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:51,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:51,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 4B21243D5AAA45058FADED9495728FC9 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:51,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 4B21243D5AAA45058FADED9495728FC9
2016-05-17 22:36:51,049 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:51,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 4B21243D5AAA45058FADED9495728FC9
2016-05-17 22:36:51,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:51,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:51,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:51,050 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524610850_0001_1_00_000000_0
2016-05-17 22:36:51,051 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:51,052 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7ba4e3ed
2016-05-17 22:36:51,053 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:51,053 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 4B21243D5AAA45058FADED9495728FC9
2016-05-17 22:36:51,053 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_ARGS_REPLACE"], id: 15D6E6686FC74758872A3763D6673E6B
2016-05-17 22:36:51,054 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_ARGS_REPLACE"]
2016-05-17 22:36:51,054 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:51,054 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:51,054 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7ba4e3ed
2016-05-17 22:36:51,054 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:51,055 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:51,055 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D3E3A626697C4596980B621EAE6C24AB, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:51,055 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:51,056 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:51,057 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:51,057 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:51,058 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524610850_0001_1_00_000000_0
2016-05-17 22:36:51,059 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524610850_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:51,060 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524610850_0001_1_00_000000_0
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:51,061 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=329777152, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524611055, Process_Duration=5, Process_End_Time=1463524611060, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:36:51,061 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D3E3A626697C4596980B621EAE6C24AB, mem on close (mb), free: 120, total: 314, max: 672
2016-05-17 22:36:51,061 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=D3E3A626697C4596980B621EAE6C24AB, index=0
2016-05-17 22:36:51,061 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610850_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D3E3A626697C4596980B621EAE6C24AB, taskAttemptId=attempt_1463524610850_0001_1_00_000000_0, startTime=1463524611027, finishTime=1463524611061, timeTaken=34, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=329777152, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524611055, Process_Duration=5, Process_End_Time=1463524611060, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:36:51,061 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=D3E3A626697C4596980B621EAE6C24AB, sourceVertex=4B21243D5AAA45058FADED9495728FC9
2016-05-17 22:36:51,061 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524610850_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:36:51,062 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524610850_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:36:51,062 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:51,062 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610850_0001_1][Event:TASK_FINISHED]: vertexName=D3E3A626697C4596980B621EAE6C24AB, taskId=task_1463524610850_0001_1_00_000000, startTime=1463524611027, finishTime=1463524611062, timeTaken=35, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
], counters=Counters: 0
2016-05-17 22:36:51,062 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524610850_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:36:51,062 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:51,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=D3E3A626697C4596980B621EAE6C24AB, sourceVertex=15D6E6686FC74758872A3763D6673E6B
2016-05-17 22:36:51,062 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB] : 1
2016-05-17 22:36:51,062 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:51,062 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB] because task failed: task_1463524610850_0001_1_00_000000
2016-05-17 22:36:51,062 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:36:51,063 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:36:51,063 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB]
2016-05-17 22:36:51,063 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=15D6E6686FC74758872A3763D6673E6B, vertexId=vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB]
2016-05-17 22:36:51,064 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:51,064 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610850_0001_1][Event:VERTEX_FINISHED]: vertexName=D3E3A626697C4596980B621EAE6C24AB, vertexId=vertex_1463524610850_0001_1_00, initRequestedTime=1463524611009, initedTime=1463524611018, startRequestedTime=1463524611019, startedTime=1463524611019, finishTime=1463524611064, timeTaken=45, status=FAILED, diagnostics=Task failed, taskId=task_1463524610850_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524611027, firstTasksToStart=[ task_1463524610850_0001_1_00_000000 ], lastTaskFinishTime=1463524611061, lastTasksToFinish=[ task_1463524610850_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:36:51,064 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:36:51,064 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610850_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524610850_0001_00_000001, stoppedTime=1463524611064, exitStatus=0
2016-05-17 22:36:51,064 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524610850_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:51,065 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524610850_0001_1_00_000000_0 failed: true container: container_1463524610850_0001_00_000001 numFailedTAs: 0
2016-05-17 22:36:51,065 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:36:51,065 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:36:51,065 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524610850_0001_1
2016-05-17 22:36:51,065 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 15D6E6686FC74758872A3763D6673E6B for vertex: vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB]
2016-05-17 22:36:51,064 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:51,066 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524610850_0001_1, queueSize=0
2016-05-17 22:36:51,066 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:51,066 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524610850_0001_00_000001
2016-05-17 22:36:51,066 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524610850_0001_00_000001
2016-05-17 22:36:51,066 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524610850_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524610850_0001_1, startTime=1463524611008, finishTime=1463524611065, timeTaken=57, status=FAILED, diagnostics=Vertex failed, vertexName=D3E3A626697C4596980B621EAE6C24AB, vertexId=vertex_1463524610850_0001_1_00, diagnostics=[Task failed, taskId=task_1463524610850_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524610850_0001_1_00 [D3E3A626697C4596980B621EAE6C24AB] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:36:51,067 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524610850_0001_1 finished with state: FAILED
2016-05-17 22:36:51,067 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524610850_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:51,067 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524610850_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:36:51 Completed Dag: dag_1463524610850_0001_1
2016-05-17 22:36:51,068 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524610850_0001_1, dagState=FAILED
2016-05-17 22:36:51,068 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524610850_0001_1_post
2016-05-17 22:36:51,068 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:51,068 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:51,068 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524610850_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:36:51,069 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524610850_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:36:51,069 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524610850_0001_00_000001 completed but with errors
2016-05-17 22:36:51,070 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524610850_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:36:51,070 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524610850_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:36:51,070 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524610850_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:36:51,071 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/ALL_ARGS_REPLACE, applicationId=application_1463524610850_0001
2016-05-17 22:36:51,073 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:51,073 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:51,073 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:51,073 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:51,073 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:51,074 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:51,074 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:51,074 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:51,074 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:51,075 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:51,075 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:51,075 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:51,076 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:51,079 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5DA426B46D4645298F74335B2AA9A24D/DD27B7B50F0D4C209BAC2BC818D683B9/.tez/application_1463524610850_0001
2016-05-17 22:36:51,079 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping all jobs
2016-05-17 22:36:51,079 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping: (1/1) ...platform/ALL_ARGS_REPLACE
2016-05-17 22:36:51,080 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopped all jobs
2016-05-17 22:36:51,080 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_ARGS_REPLACE/_temporary
2016-05-17 22:36:51,081 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.248
2016-05-17 22:36:51,081 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_ARGS_REPLACE
2016-05-17 22:36:51,111 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:36:51,111 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:36:51,141 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.004
2016-05-17 22:36:51,146 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 22:36:51,146 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:51,147 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:36:51,147 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:36:51,160 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:36:51,161 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_ARGS_SWAP
2016-05-17 22:36:51,168 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting
2016-05-17 22:36:51,169 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:51,169 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_ARGS_SWAP"]
2016-05-17 22:36:51,169 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] parallel execution of steps is enabled: false
2016-05-17 22:36:51,169 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] executing total steps: 1
2016-05-17 22:36:51,169 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] allocating management threads: 1
2016-05-17 22:36:51,169 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] starting step: (1/1) ...blyplatform/ALL_ARGS_SWAP
2016-05-17 22:36:51,169 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [each] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:36:51,180 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] tez session mode enabled: true
2016-05-17 22:36:51,184 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:51,184 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:36:51,184 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:36:51,193 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B6B0D42EA9994697AC5445D51ACAF442/F2E55A54E8494660A45502B5F331595E/.tez/application_1463524611184_0001 doesn't exist and is created
2016-05-17 22:36:51,207 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:36:51,207 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:36:51,208 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B6B0D42EA9994697AC5445D51ACAF442/F2E55A54E8494660A45502B5F331595E/.tez/application_1463524611184_0001
2016-05-17 22:36:51,210 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524611184_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:36:51,214 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:36:51,214 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:36:51,230 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:36:51,230 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:36:51,231 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:36:51,231 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:36:51,232 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B6B0D42EA9994697AC5445D51ACAF442/F2E55A54E8494660A45502B5F331595E/.tez/application_1463524611184_0001/localmode-log-dir
2016-05-17 22:36:51,233 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B6B0D42EA9994697AC5445D51ACAF442/F2E55A54E8494660A45502B5F331595E/.tez/application_1463524611184_0001/localmode-log-dir/history.txt.appattempt_1463524611184_0001_000000, maxErrors=10
2016-05-17 22:36:51,233 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:36:51,233 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524611184_0001_000000, appSubmitTime=1463524611209, launchTime=1463524611209
2016-05-17 22:36:51,239 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:46837
2016-05-17 22:36:51,239 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:36:51,239 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:36:51,240 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:36:51,240 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:36:51,241 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:36:51,242 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524611184_0001_000000, startTime=1463524611242
2016-05-17 22:36:51,242 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:36:51,243 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:36:51,308 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:36:51,308 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:36:51,308 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/ALL_ARGS_SWAP, applicationId=application_1463524611184_0001, dagName=[B6B0D42EA9994697AC5445D51ACAF442/F2E55A54E8494660A45502B5F331595E] each/(1/1) ...blyplatform/ALL_ARGS_SWAP
2016-05-17 22:36:51,322 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [B6B0D42EA9994697AC5445D51ACAF442/F2E55A54E8494660A45502B5F331595E] each/(1/1) ...blyplatform/ALL_ARGS_SWAP
2016-05-17 22:36:51,323 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524611184_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B6B0D42EA9994697AC5445D51ACAF442/F2E55A54E8494660A45502B5F331595E/.tez/application_1463524611184_0001/localmode-log-dir/dag_1463524611184_0001_1.dot
2016-05-17 22:36:51,323 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B6B0D42EA9994697AC5445D51ACAF442/F2E55A54E8494660A45502B5F331595E/.tez/application_1463524611184_0001/localmode-log-dir/dag_1463524611184_0001_1-tez-dag.pb.txt
2016-05-17 22:36:51,327 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524611184_0001_1
2016-05-17 22:36:51,327 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:51,327 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [B6B0D42EA9994697AC5445D51ACAF442/F2E55A54E8494660A45502B5F331595E] each/(1/1) ...blyplatform/ALL_ARGS_SWAP
2016-05-17 22:36:51 Running Dag: dag_1463524611184_0001_1
2016-05-17 22:36:51,338 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524611184_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524611184_0001_1, submitTime=1463524611322
2016-05-17 22:36:51,338 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:36:51,339 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CE1002CF9F9D4A30BD4AA06751277CAA
2016-05-17 22:36:51,339 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:36:51,339 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524611184_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524611184_0001_1, initTime=1463524611338
2016-05-17 22:36:51,339 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:36:51,340 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524611184_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:36:51,340 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524611184_0001_1][Event:DAG_STARTED]: dagID=dag_1463524611184_0001_1, startTime=1463524611340
2016-05-17 22:36:51,340 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:36:51,340 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:36:51,341 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524611184_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:36:51,341 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/ALL_ARGS_SWAP, applicationId=application_1463524611184_0001, dagName=[B6B0D42EA9994697AC5445D51ACAF442/F2E55A54E8494660A45502B5F331595E] each/(1/1) ...blyplatform/ALL_ARGS_SWAP
2016-05-17 22:36:51,341 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CE1002CF9F9D4A30BD4AA06751277CAA : {C81742C61D3342BAB6EED1AB5BE796FD={InputName=C81742C61D3342BAB6EED1AB5BE796FD}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:36:51,341 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: C81742C61D3342BAB6EED1AB5BE796FD, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:36:51,341 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA]
2016-05-17 22:36:51,341 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA]
2016-05-17 22:36:51,341 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA]
2016-05-17 22:36:51,341 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA]. Starting root input initializers: 1
2016-05-17 22:36:51,342 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:36:51,343 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: C81742C61D3342BAB6EED1AB5BE796FD on vertex vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA]
2016-05-17 22:36:51,345 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:36:51,345 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input C81742C61D3342BAB6EED1AB5BE796FD asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:36:51,345 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:36:51,346 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:36:51,347 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:36:51,347 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: C81742C61D3342BAB6EED1AB5BE796FD on vertex vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA]
2016-05-17 22:36:51,348 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:51,348 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {C81742C61D3342BAB6EED1AB5BE796FD=forAllWorkUnits=true, update=[1]}
2016-05-17 22:36:51,348 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA] parallelism set to 1
2016-05-17 22:36:51,348 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:36:51,348 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524611184_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524611184_0001_1_00, eventCount=1
2016-05-17 22:36:51,349 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA]
2016-05-17 22:36:51,349 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=908B751E7B1A444CA9AC418598EB697A, vertexId=vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:36:51,349 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=908B751E7B1A444CA9AC418598EB697A, vertexId=vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA]
2016-05-17 22:36:51,350 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=908B751E7B1A444CA9AC418598EB697A, vertexName=CE1002CF9F9D4A30BD4AA06751277CAA, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:36:51,351 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=908B751E7B1A444CA9AC418598EB697A, vertexId=vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA]
2016-05-17 22:36:51,352 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524611184_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CE1002CF9F9D4A30BD4AA06751277CAA, vertexId=vertex_1463524611184_0001_1_00, initRequestedTime=1463524611341, initedTime=1463524611352, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:36:51,352 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA] with distanceFromRoot: 0
2016-05-17 22:36:51,352 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:36:51,352 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:36:51,352 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CE1002CF9F9D4A30BD4AA06751277CAA
2016-05-17 22:36:51,352 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA]
2016-05-17 22:36:51,353 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524611184_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524611184_0001_1_00, startRequestedTime=1463524611352, startedTime=1463524611352
2016-05-17 22:36:51,353 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:36:51,353 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:36:51,353 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:36:51,353 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524611184_0001_1][Event:TASK_STARTED]: vertexName=CE1002CF9F9D4A30BD4AA06751277CAA, taskId=task_1463524611184_0001_1_00_000000, scheduledTime=1463524611353, launchTime=1463524611353
2016-05-17 22:36:51,354 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524611184_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:36:51,354 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA]
2016-05-17 22:36:51,354 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B6B0D42EA9994697AC5445D51ACAF442/F2E55A54E8494660A45502B5F331595E] each/(1/1) ...blyplatform/ALL_ARGS_SWAP, VertexName: CE1002CF9F9D4A30BD4AA06751277CAA, VertexParallelism: 1, TaskAttemptID:attempt_1463524611184_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C81742C61D3342BAB6EED1AB5BE796FD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=908B751E7B1A444CA9AC418598EB697A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:51,354 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524611184_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:36:51,354 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:36:51,355 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:36:51,355 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:36:51,355 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:36:51,356 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524611184_0001_00_000001
2016-05-17 22:36:51,356 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524611184_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:36:51,357 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] submitted tez dag to app master: application_1463524611184_0001, with dag id: dag_1463524611184_0001_1
2016-05-17 22:36:51,358 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524611184_0001_00_000001
2016-05-17 22:36:51,358 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:36:51,359 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524611184_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:36:51,359 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:36:51,359 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524611184_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524611184_0001_00_000001, launchTime=1463524611359
2016-05-17 22:36:51,360 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524611184_0001_00_000001
2016-05-17 22:36:51,360 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524611184_0001_1_00_000000_0] to container: [container_1463524611184_0001_00_000001]
2016-05-17 22:36:51,360 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524611184_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:36:51,361 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524611184_0001_1_00_000000_0] started. Is using containerId: [container_1463524611184_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:36:51,361 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:36:51,361 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524611184_0001_00_000001 given task: attempt_1463524611184_0001_1_00_000000_0
2016-05-17 22:36:51,361 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524611184_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CE1002CF9F9D4A30BD4AA06751277CAA, taskAttemptId=attempt_1463524611184_0001_1_00_000000_0, startTime=1463524611360, containerId=container_1463524611184_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524611184_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:36:51,361 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524611184_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:36:51,361 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524611184_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524611184_0001_1_00_000000_0
2016-05-17 22:36:51,361 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524611184_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:36:51 Starting to run new task attempt: attempt_1463524611184_0001_1_00_000000_0
2016-05-17 22:36:51,362 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524611184_0001_1_00_000000_0
2016-05-17 22:36:51,362 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:51,362 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:36:51,362 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:36:51,362 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:36:51,362 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:36:51,362 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:36:51,362 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B6B0D42EA9994697AC5445D51ACAF442/F2E55A54E8494660A45502B5F331595E] each/(1/1) ...blyplatform/ALL_ARGS_SWAP, VertexName: CE1002CF9F9D4A30BD4AA06751277CAA, VertexParallelism: 1, TaskAttemptID:attempt_1463524611184_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C81742C61D3342BAB6EED1AB5BE796FD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=908B751E7B1A444CA9AC418598EB697A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:36:51,363 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:36:51,363 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524611184_0001_1_00_000000_0
2016-05-17 22:36:51,363 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:36:51,363 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:51,363 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524611184_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:36:51,364 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:51,364 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=908B751E7B1A444CA9AC418598EB697A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:36:51,364 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:36:51,364 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 908B751E7B1A444CA9AC418598EB697A
2016-05-17 22:36:51,364 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:36:51,364 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 908B751E7B1A444CA9AC418598EB697A
2016-05-17 22:36:51,365 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C81742C61D3342BAB6EED1AB5BE796FD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:36:51,366 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:36:51,366 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:36:51,366 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C81742C61D3342BAB6EED1AB5BE796FD
2016-05-17 22:36:51,367 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: C81742C61D3342BAB6EED1AB5BE796FD
2016-05-17 22:36:51,369 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:36:51,369 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: C81742C61D3342BAB6EED1AB5BE796FD
2016-05-17 22:36:51,369 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C81742C61D3342BAB6EED1AB5BE796FD
2016-05-17 22:36:51,372 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CE1002CF9F9D4A30BD4AA06751277CAA, ordinal: 0
2016-05-17 22:36:51,372 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CE1002CF9F9D4A30BD4AA06751277CAA, mem on start (mb), free: 188, total: 300, max: 672
2016-05-17 22:36:51,373 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:36:51,373 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:36:51,373 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:36:51,374 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:36:51,374 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 908B751E7B1A444CA9AC418598EB697A
2016-05-17 22:36:51,374 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:36:51,374 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:36:51,375 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:36:51,375 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:51,375 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:36:51,375 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:36:51,375 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:36:51,376 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:36:51,376 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 908B751E7B1A444CA9AC418598EB697A, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:36:51,376 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C81742C61D3342BAB6EED1AB5BE796FD, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:36:51,376 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:36:51,376 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C81742C61D3342BAB6EED1AB5BE796FD being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:36:51,376 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:36:51,376 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:36:51,376 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C81742C61D3342BAB6EED1AB5BE796FD
2016-05-17 22:36:51,377 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:51,377 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C81742C61D3342BAB6EED1AB5BE796FD
2016-05-17 22:36:51,377 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:36:51,377 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524611184_0001_1_00_000000_0
2016-05-17 22:36:51,377 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:36:51,378 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@19e1bfba
2016-05-17 22:36:51,379 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@19e1bfba
2016-05-17 22:36:51,379 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:36:51,379 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:36:51,380 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:51,380 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: C81742C61D3342BAB6EED1AB5BE796FD
2016-05-17 22:36:51,380 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_ARGS_SWAP"], id: 908B751E7B1A444CA9AC418598EB697A
2016-05-17 22:36:51,380 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_ARGS_SWAP"]
2016-05-17 22:36:51,380 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:36:51,381 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:36:51,381 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CE1002CF9F9D4A30BD4AA06751277CAA, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:36:51,381 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:51,381 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:51,382 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:36:51,382 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:36:51,382 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524611184_0001_1_00_000000_0
2016-05-17 22:36:51,382 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524611184_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:36:51,383 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524611184_0001_1_00_000000_0
cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
2016-05-17 22:36:51,384 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:36:51,384 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524611184_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CE1002CF9F9D4A30BD4AA06751277CAA, taskAttemptId=attempt_1463524611184_0001_1_00_000000_0, startTime=1463524611360, finishTime=1463524611384, timeTaken=24, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=315097088, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524611381, Process_Duration=2, Process_End_Time=1463524611383, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:36:51,384 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524611184_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:36:51,385 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524611184_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:36:51,385 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:36:51,385 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524611184_0001_1][Event:TASK_FINISHED]: vertexName=CE1002CF9F9D4A30BD4AA06751277CAA, taskId=task_1463524611184_0001_1_00_000000, startTime=1463524611360, finishTime=1463524611385, timeTaken=25, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
], counters=Counters: 0
2016-05-17 22:36:51,385 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524611184_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:36:51,385 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA] : 1
2016-05-17 22:36:51,385 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA] because task failed: task_1463524611184_0001_1_00_000000
2016-05-17 22:36:51,385 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:36:51,385 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:36:51,385 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:36:51,385 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA]
2016-05-17 22:36:51,385 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=908B751E7B1A444CA9AC418598EB697A, vertexId=vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA]
2016-05-17 22:36:51,386 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:36:51,386 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524611184_0001_1][Event:VERTEX_FINISHED]: vertexName=CE1002CF9F9D4A30BD4AA06751277CAA, vertexId=vertex_1463524611184_0001_1_00, initRequestedTime=1463524611341, initedTime=1463524611352, startRequestedTime=1463524611352, startedTime=1463524611352, finishTime=1463524611386, timeTaken=34, status=FAILED, diagnostics=Task failed, taskId=task_1463524611184_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524611360, firstTasksToStart=[ task_1463524611184_0001_1_00_000000 ], lastTaskFinishTime=1463524611384, lastTasksToFinish=[ task_1463524611184_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:36:51,386 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:36:51,388 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524611184_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524611184_0001_00_000001, stoppedTime=1463524611388, exitStatus=0
2016-05-17 22:36:51,388 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524611184_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:36:51,389 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524611184_0001_1_00_000000_0 failed: true container: container_1463524611184_0001_00_000001 numFailedTAs: 0
2016-05-17 22:36:51,389 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:36:51,389 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:36:51,389 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524611184_0001_1
2016-05-17 22:36:51,389 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 908B751E7B1A444CA9AC418598EB697A for vertex: vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA]
2016-05-17 22:36:51,388 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:36:51,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 19 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=315097088, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524611381, Process_Duration=2, Process_End_Time=1463524611383, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:36:51,389 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524611184_0001_1, queueSize=0
2016-05-17 22:36:51,390 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:36:51,390 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524611184_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524611184_0001_1, startTime=1463524611340, finishTime=1463524611389, timeTaken=49, status=FAILED, diagnostics=Vertex failed, vertexName=CE1002CF9F9D4A30BD4AA06751277CAA, vertexId=vertex_1463524611184_0001_1_00, diagnostics=[Task failed, taskId=task_1463524611184_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [each][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524611184_0001_1_00 [CE1002CF9F9D4A30BD4AA06751277CAA] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:36:51,390 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524611184_0001_1 finished with state: FAILED
2016-05-17 22:36:51,390 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524611184_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:36:51,389 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CE1002CF9F9D4A30BD4AA06751277CAA, mem on close (mb), free: 185, total: 300, max: 672
2016-05-17 22:36:51,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=CE1002CF9F9D4A30BD4AA06751277CAA, index=0
2016-05-17 22:36:51,390 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524611184_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:36:51 Completed Dag: dag_1463524611184_0001_1
2016-05-17 22:36:51,390 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524611184_0001_1, dagState=FAILED
2016-05-17 22:36:51,390 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524611184_0001_1_post
2016-05-17 22:36:51,390 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:36:51,390 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:36:51,391 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524611184_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:36:51,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=CE1002CF9F9D4A30BD4AA06751277CAA, sourceVertex=C81742C61D3342BAB6EED1AB5BE796FD
2016-05-17 22:36:51,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=CE1002CF9F9D4A30BD4AA06751277CAA, sourceVertex=908B751E7B1A444CA9AC418598EB697A
2016-05-17 22:36:51,391 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524611184_0001_00_000001
2016-05-17 22:36:51,391 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524611184_0001_00_000001
2016-05-17 22:36:51,392 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524611184_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:36:51,392 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524611184_0001_00_000001 completed but with errors
2016-05-17 22:36:51,392 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524611184_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:36:51,392 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524611184_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:36:51,392 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524611184_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:36:51,401 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/ALL_ARGS_SWAP, applicationId=application_1463524611184_0001
2016-05-17 22:36:51,402 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:36:51,402 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:36:51,402 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:36:51,402 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:36:51,403 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:36:51,403 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:36:51,403 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:36:51,403 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:36:51,403 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:36:51,404 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:36:51,404 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:36:51,404 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:36:51,404 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:36:51,407 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B6B0D42EA9994697AC5445D51ACAF442/F2E55A54E8494660A45502B5F331595E/.tez/application_1463524611184_0001
2016-05-17 22:36:51,407 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping all jobs
2016-05-17 22:36:51,407 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopping: (1/1) ...blyplatform/ALL_ARGS_SWAP
2016-05-17 22:36:51,408 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] stopped all jobs
2016-05-17 22:36:51,408 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/eachpipeassemblyplatform/ALL_ARGS_SWAP/_temporary
2016-05-17 22:36:51,408 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [each] completed in: 00:00.240
2016-05-17 22:36:51,408 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/eachpipeassemblyplatform//ALL_ARGS_SWAP