2016-05-18 00:57:56,637 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(197)) - found 1 test platforms from classpath
2016-05-18 00:57:56,669 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(246)) - adding test: cascading.detail.EveryEveryPipeAssemblyPlatformTest, with platform: hadoop2-tez
2016-05-18 00:57:57,236 INFO tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-05-18 00:57:57,725 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-05-18 00:57:58,078 INFO platform.TestPlatform (TestPlatform.java:getGlobalProperties(75)) - platform property overrides:
2016-05-18 00:57:58,364 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:57:58,365 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:57:58,486 INFO property.AppProps (AppProps.java:getAppID(177)) - using app.id: BBCC2D1309B14118B182802D3FA99BAD
2016-05-18 00:57:58,650 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.064
2016-05-18 00:57:58,664 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.078
2016-05-18 00:57:58,666 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:57:58,666 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:57:58,667 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:57:58,843 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:57:58,846 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_-1
2016-05-18 00:57:59,065 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.1.0
2016-05-18 00:57:59,068 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:57:59,069 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:57:59,069 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_-1"]
2016-05-18 00:57:59,070 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:57:59,070 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:57:59,071 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:57:59,072 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...d'_RESULTS.-1_'field2'_-1
2016-05-18 00:57:59,072 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:57:59,118 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:57:59,311 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:57:59,322 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:57:59,322 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:57:59,399 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8BD898ADBBE948D48CE964B7664781F6/EBEA34E6FEA943A79DDF2AD83F0CB9DB/.tez/application_1463533079321_0001 doesn't exist and is created
2016-05-18 00:57:59,503 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:57:59,503 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:57:59,505 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8BD898ADBBE948D48CE964B7664781F6/EBEA34E6FEA943A79DDF2AD83F0CB9DB/.tez/application_1463533079321_0001_wd
2016-05-18 00:57:59,565 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533079321_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:57:59,579 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:57:59,580 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:57:59,590 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:57:59,590 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:57:59,590 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:57:59,591 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:57:59,604 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-05-18 00:57:59,604 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-05-18 00:57:59,604 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:57:59,614 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:57:59,639 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:57:59,640 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:57:59,643 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:57:59,644 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:57:59,644 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:57:59,645 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:57:59,646 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:57:59,647 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:57:59,649 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:57:59,649 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:57:59,657 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:57:59,661 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:57:59,680 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:57:59,682 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:57:59,682 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:57:59,686 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8BD898ADBBE948D48CE964B7664781F6/EBEA34E6FEA943A79DDF2AD83F0CB9DB/.tez/application_1463533079321_0001_wd/localmode-log-dir
2016-05-18 00:57:59,687 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8BD898ADBBE948D48CE964B7664781F6/EBEA34E6FEA943A79DDF2AD83F0CB9DB/.tez/application_1463533079321_0001_wd/localmode-log-dir/history.txt.appattempt_1463533079321_0001_000000, maxErrors=10
2016-05-18 00:57:59,688 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8BD898ADBBE948D48CE964B7664781F6/EBEA34E6FEA943A79DDF2AD83F0CB9DB/.tez/application_1463533079321_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:57:59,689 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533079321_0001_000000, appSubmitTime=1463533079545, launchTime=1463533079563
2016-05-18 00:57:59,704 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:57:59,705 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-18 00:57:59,707 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-18 00:57:59,707 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:57:59,768 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49058
2016-05-18 00:57:59,770 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533079321
2016-05-18 00:57:59,785 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533079321_0001_000000, startTime=1463533079784
2016-05-18 00:57:59,785 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:57:59,807 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:57:59,823 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:57:59,823 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...d'_RESULTS.-1_'field2'_-1, applicationId=application_1463533079321_0001, dagName=[8BD898ADBBE948D48CE964B7664781F6/EBEA34E6FEA943A79DDF2AD83F0CB9DB] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1
2016-05-18 00:58:00,051 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [8BD898ADBBE948D48CE964B7664781F6/EBEA34E6FEA943A79DDF2AD83F0CB9DB] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1
2016-05-18 00:58:00,060 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:00,126 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533079321_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8BD898ADBBE948D48CE964B7664781F6/EBEA34E6FEA943A79DDF2AD83F0CB9DB/.tez/application_1463533079321_0001_wd/localmode-log-dir/dag_1463533079321_0001_1.dot
2016-05-18 00:58:00,130 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8BD898ADBBE948D48CE964B7664781F6/EBEA34E6FEA943A79DDF2AD83F0CB9DB/.tez/application_1463533079321_0001_wd/localmode-log-dir/dag_1463533079321_0001_1-tez-dag.pb.txt
2016-05-18 00:58:00,169 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533079321_0001_1
2016-05-18 00:58:00,169 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:00,169 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [8BD898ADBBE948D48CE964B7664781F6/EBEA34E6FEA943A79DDF2AD83F0CB9DB] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1
2016-05-18 00:58:00 Running Dag: dag_1463533079321_0001_1
2016-05-18 00:58:00,194 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533079321_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533079321_0001_1, submitTime=1463533080051
2016-05-18 00:58:00,250 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8BD898ADBBE948D48CE964B7664781F6/EBEA34E6FEA943A79DDF2AD83F0CB9DB/.tez/application_1463533079321_0001/tez-conf.pb" } size: 36913 timestamp: 1463533079000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:00,251 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0]
2016-05-18 00:58:00,251 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0]
2016-05-18 00:58:00,252 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:00,253 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:00,256 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533079321_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533079321_0001_1, initTime=1463533080255
2016-05-18 00:58:00,257 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533079321_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:00,261 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:00,265 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533079321_0001_1][Event:DAG_STARTED]: dagID=dag_1463533079321_0001_1, startTime=1463533080264
2016-05-18 00:58:00,265 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533079321_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:00,265 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: A160DD092D674A239301225213511FE0 : {6545F2699D9C463D9324684E0F6A368F={InputName=6545F2699D9C463D9324684E0F6A368F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:00,268 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 6545F2699D9C463D9324684E0F6A368F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:00,268 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0]
2016-05-18 00:58:00,273 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...d'_RESULTS.-1_'field2'_-1, applicationId=application_1463533079321_0001, dagName=[8BD898ADBBE948D48CE964B7664781F6/EBEA34E6FEA943A79DDF2AD83F0CB9DB] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1
2016-05-18 00:58:00,280 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0]
2016-05-18 00:58:00,280 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0]
2016-05-18 00:58:00,282 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0]
2016-05-18 00:58:00,297 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 6545F2699D9C463D9324684E0F6A368F on vertex vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0]
2016-05-18 00:58:00,303 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:00,310 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533079321_0001, with dag id: dag_1463533079321_0001_1
2016-05-18 00:58:00,325 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 6545F2699D9C463D9324684E0F6A368F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:00,359 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:00,360 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:00,361 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:00,362 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:00,373 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 6545F2699D9C463D9324684E0F6A368F on vertex vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0]
2016-05-18 00:58:00,382 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:00,382 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {6545F2699D9C463D9324684E0F6A368F=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:00,399 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0] parallelism set to 1
2016-05-18 00:58:00,400 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:00,409 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0], numAdditionalOutputs=1
2016-05-18 00:58:00,409 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=CE577A9C9CD941B899622E440291A295, vertex=vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:00,418 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for A160DD092D674A239301225213511FE0:CE577A9C9CD941B899622E440291A295 using oldmapred API
2016-05-18 00:58:00,419 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=CE577A9C9CD941B899622E440291A295, vertexName=A160DD092D674A239301225213511FE0, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:00,429 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533079321_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A160DD092D674A239301225213511FE0, vertexId=vertex_1463533079321_0001_1_00, initRequestedTime=1463533080265, initedTime=1463533080409, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:00,429 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0] with distanceFromRoot: 0
2016-05-18 00:58:00,429 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:00,431 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533079321_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533079321_0001_1_00, startRequestedTime=1463533080303, startedTime=1463533080429
2016-05-18 00:58:00,432 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533079321_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533079321_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:00,433 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:00,434 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in A160DD092D674A239301225213511FE0
2016-05-18 00:58:00,434 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0]
2016-05-18 00:58:00,439 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533079321_0001_1][Event:TASK_STARTED]: vertexName=A160DD092D674A239301225213511FE0, taskId=task_1463533079321_0001_1_00_000000, scheduledTime=1463533080438, launchTime=1463533080438
2016-05-18 00:58:00,462 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0]
2016-05-18 00:58:00,506 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:00,523 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533079321_0001_00_000001, launchTime=1463533080522
2016-05-18 00:58:00,529 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533079321_0001_00_000001
2016-05-18 00:58:00,532 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533079321_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533079321_0001_1_00_000000_0
2016-05-18 00:58:00 Starting to run new task attempt: attempt_1463533079321_0001_1_00_000000_0
2016-05-18 00:58:00,534 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533079321_0001_1_00_000000_0
2016-05-18 00:58:00,535 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533079321_0001_1_00_000000_0] started. Is using containerId: [container_1463533079321_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:00,536 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:00,538 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:00,540 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533079321_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A160DD092D674A239301225213511FE0, taskAttemptId=attempt_1463533079321_0001_1_00_000000_0, startTime=1463533080532, containerId=container_1463533079321_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:00,541 INFO counters.Limits (Limits.java:ensureInitialized(60)) - Counter limits initialized with parameters: GROUP_NAME_MAX=256, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-05-18 00:58:00,552 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8BD898ADBBE948D48CE964B7664781F6/EBEA34E6FEA943A79DDF2AD83F0CB9DB] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1, VertexName: A160DD092D674A239301225213511FE0, VertexParallelism: 1, TaskAttemptID:attempt_1463533079321_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6545F2699D9C463D9324684E0F6A368F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CE577A9C9CD941B899622E440291A295, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:00,554 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:00,558 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533079321_0001_1_00_000000_0
2016-05-18 00:58:00,563 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533079321_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:00,564 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533079321_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:00,598 INFO input.MRInput (MRInput.java:initialize(440)) - 6545F2699D9C463D9324684E0F6A368F using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:00,600 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 6545F2699D9C463D9324684E0F6A368F
2016-05-18 00:58:00,601 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:00,612 INFO output.MROutput (MROutput.java:initialize(421)) - CE577A9C9CD941B899622E440291A295: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:00,618 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A160DD092D674A239301225213511FE0, ordinal: 0
2016-05-18 00:58:00,618 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A160DD092D674A239301225213511FE0, mem on start (mb), free: 173, total: 281, max: 672
2016-05-18 00:58:00,619 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:00,619 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:00,619 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:00,620 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:00,620 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[6545F2699D9C463D9324684E0F6A368F:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [CE577A9C9CD941B899622E440291A295:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:00,624 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:00,627 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:00,628 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:00,628 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[6545F2699D9C463D9324684E0F6A368F:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [CE577A9C9CD941B899622E440291A295:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:00,628 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:00,629 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 6545F2699D9C463D9324684E0F6A368F being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:00,629 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 6545F2699D9C463D9324684E0F6A368F
2016-05-18 00:58:00,629 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:00,630 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:00,630 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:00,631 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533079321_0001_1_00_000000_0
2016-05-18 00:58:00,651 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:00,652 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 6545F2699D9C463D9324684E0F6A368F initialized RecordReader from event
2016-05-18 00:58:00,670 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:00,694 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 6545F2699D9C463D9324684E0F6A368F
2016-05-18 00:58:00,694 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_-1"], id: CE577A9C9CD941B899622E440291A295
2016-05-18 00:58:00,694 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_-1"]
2016-05-18 00:58:00,698 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:00,700 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A160DD092D674A239301225213511FE0, all 1 inputs ready in: 00:00:00.001
2016-05-18 00:58:00,710 INFO output.MROutput (MROutput.java:close(526)) - CE577A9C9CD941B899622E440291A295 closed
2016-05-18 00:58:00,712 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533079321_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:00,713 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533079321_0001_1_00_000000_0
2016-05-18 00:58:00,713 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A160DD092D674A239301225213511FE0, mem on close (mb), free: 165, total: 281, max: 672
2016-05-18 00:58:00,714 INFO output.MROutput (MROutput.java:close(526)) - CE577A9C9CD941B899622E440291A295 closed
2016-05-18 00:58:00,714 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:00 Completed running task attempt: attempt_1463533079321_0001_1_00_000000_0
2016-05-18 00:58:00,716 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533079321_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:00,716 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533079321_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:00,720 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533079321_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533080701, Process_Duration=12, Process_End_Time=1463533080713, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:00,725 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533079321_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:00,725 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:00,727 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:00,727 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:00,727 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533079321_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A160DD092D674A239301225213511FE0, taskAttemptId=attempt_1463533079321_0001_1_00_000000_0, creationTime=1463533080455, allocationTime=1463533080514, startTime=1463533080532, finishTime=1463533080725, timeTaken=193, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:00,728 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533079321_0001_00_000001
2016-05-18 00:58:00,730 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533079321_0001_1_00_000000_0
2016-05-18 00:58:00,741 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533079321_0001_1][Event:TASK_FINISHED]: vertexName=A160DD092D674A239301225213511FE0, taskId=task_1463533079321_0001_1_00_000000, startTime=1463533080532, finishTime=1463533080731, timeTaken=199, status=SUCCEEDED, successfulAttemptID=attempt_1463533079321_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533080701, Process_Duration=12, Process_End_Time=1463533080713, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:00,752 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:00,753 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0]
2016-05-18 00:58:00,771 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533079321_0001_1][Event:VERTEX_FINISHED]: vertexName=A160DD092D674A239301225213511FE0, vertexId=vertex_1463533079321_0001_1_00, initRequestedTime=1463533080265, initedTime=1463533080409, startRequestedTime=1463533080303, startedTime=1463533080429, finishTime=1463533080754, timeTaken=325, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533080701, Process_Duration=12, Process_End_Time=1463533080713, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533080532, firstTasksToStart=[ task_1463533079321_0001_1_00_000000 ], lastTaskFinishTime=1463533080725, lastTasksToFinish=[ task_1463533079321_0001_1_00_000000 ], minTaskDuration=193, maxTaskDuration=193, avgTaskDuration=193.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533079321_0001_1_00_000000 ], longestDurationTasks=[ task_1463533079321_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:00,772 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:00,775 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533079321_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533079321_0001_00_000001, stoppedTime=1463533080775, exitStatus=0
2016-05-18 00:58:00,776 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:00,777 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:00,779 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533079321_0001_1
2016-05-18 00:58:00,780 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533079321_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533079321_0001_1
2016-05-18 00:58:00,780 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533079321_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:00,780 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: CE577A9C9CD941B899622E440291A295 for vertex: vertex_1463533079321_0001_1_00 [A160DD092D674A239301225213511FE0], outputName: CE577A9C9CD941B899622E440291A295
2016-05-18 00:58:00,781 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533079321_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:00,791 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533079321_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:00,792 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533079321_0001_00_000001: 64 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:00,792 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533079321_0001_00_000001, Exiting
2016-05-18 00:58:00,792 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533079321_0001_00_000001
2016-05-18 00:58:00,792 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533079321_0001_00_000001
2016-05-18 00:58:00,794 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533079321_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:00,794 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:CE577A9C9CD941B899622E440291A295 of vertex/vertexGroup:A160DD092D674A239301225213511FE0 isVertexGroupOutput:false
2016-05-18 00:58:00,795 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533079321_0001_00_000001 completed successfully
2016-05-18 00:58:00,795 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:00,801 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533079321_0001_1, queueSize=0
2016-05-18 00:58:00,804 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533079321_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533079321_0001_1, startTime=1463533080261, finishTime=1463533080795, timeTaken=534, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=1410, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533080701, Process_Duration=12, Process_End_Time=1463533080713, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:00,805 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533079321_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:00,805 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533079321_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:00 Completed Dag: dag_1463533079321_0001_1
2016-05-18 00:58:00,808 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533079321_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:00,809 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533079321_0001_1_post
2016-05-18 00:58:00,810 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:00,810 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:00,810 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:00,815 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[8BD898ADBBE948D48CE964B7664781F6/EBEA34E6FEA943A79DDF2AD83F0CB9DB] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1, with id=dag_1463533079321_0001_1
2016-05-18 00:58:00,820 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[8BD898ADBBE948D48CE964B7664781F6/EBEA34E6FEA943A79DDF2AD83F0CB9DB] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1, with id=dag_1463533079321_0001_1
2016-05-18 00:58:00,822 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...d'_RESULTS.-1_'field2'_-1, applicationId=application_1463533079321_0001
2016-05-18 00:58:00,828 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:00,828 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:00,829 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:00,829 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:00,829 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:00,830 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:00,832 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:00,833 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:00,833 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:00,833 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:00,833 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:00,834 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:00,834 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:00,835 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:00,841 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8BD898ADBBE948D48CE964B7664781F6/EBEA34E6FEA943A79DDF2AD83F0CB9DB/.tez/application_1463533079321_0001
2016-05-18 00:58:00,843 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_-1/_temporary
2016-05-18 00:58:00,843 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:01.773
2016-05-18 00:58:00,997 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_'field2'_-1
2016-05-18 00:58:01,038 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:01,039 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:01,107 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.031
2016-05-18 00:58:01,124 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.048
2016-05-18 00:58:01,125 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:01,125 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:01,126 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:01,147 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:01,148 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field2'
2016-05-18 00:58:01,191 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:01,191 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:01,191 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field2'"]
2016-05-18 00:58:01,192 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:01,192 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:01,192 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:01,193 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ULTS.-1_'field2'_'field2'
2016-05-18 00:58:01,194 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:01,213 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:01,218 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:01,219 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:01,219 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:01,230 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/4045B0F073F64505879CA3F11D1AC7F4/A00AB4BA69D1491693583B434F772596/.tez/application_1463533081218_0001 doesn't exist and is created
2016-05-18 00:58:01,255 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:01,255 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:01,257 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/4045B0F073F64505879CA3F11D1AC7F4/A00AB4BA69D1491693583B434F772596/.tez/application_1463533081218_0001_wd
2016-05-18 00:58:01,268 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533081218_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:01,269 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:01,269 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:01,276 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:01,276 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:01,276 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:01,276 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:01,277 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:01,281 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:01,282 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:01,282 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:01,282 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:01,282 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:01,282 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:01,282 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:01,283 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:01,283 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:01,283 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:01,284 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:01,284 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:01,289 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:01,291 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:01,291 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:01,292 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/4045B0F073F64505879CA3F11D1AC7F4/A00AB4BA69D1491693583B434F772596/.tez/application_1463533081218_0001_wd/localmode-log-dir
2016-05-18 00:58:01,293 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/4045B0F073F64505879CA3F11D1AC7F4/A00AB4BA69D1491693583B434F772596/.tez/application_1463533081218_0001_wd/localmode-log-dir/history.txt.appattempt_1463533081218_0001_000000, maxErrors=10
2016-05-18 00:58:01,293 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/4045B0F073F64505879CA3F11D1AC7F4/A00AB4BA69D1491693583B434F772596/.tez/application_1463533081218_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:01,293 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533081218_0001_000000, appSubmitTime=1463533081268, launchTime=1463533081268
2016-05-18 00:58:01,307 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:01,315 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:38943
2016-05-18 00:58:01,316 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533081218
2016-05-18 00:58:01,324 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533081218_0001_000000, startTime=1463533081324
2016-05-18 00:58:01,325 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:01,356 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:01,356 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:01,356 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ULTS.-1_'field2'_'field2', applicationId=application_1463533081218_0001, dagName=[4045B0F073F64505879CA3F11D1AC7F4/A00AB4BA69D1491693583B434F772596] every.every/(1/1) ...ULTS.-1_'field2'_'field2'
2016-05-18 00:58:01,379 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [4045B0F073F64505879CA3F11D1AC7F4/A00AB4BA69D1491693583B434F772596] every.every/(1/1) ...ULTS.-1_'field2'_'field2'
2016-05-18 00:58:01,379 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:01,380 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533081218_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/4045B0F073F64505879CA3F11D1AC7F4/A00AB4BA69D1491693583B434F772596/.tez/application_1463533081218_0001_wd/localmode-log-dir/dag_1463533081218_0001_1.dot
2016-05-18 00:58:01,382 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/4045B0F073F64505879CA3F11D1AC7F4/A00AB4BA69D1491693583B434F772596/.tez/application_1463533081218_0001_wd/localmode-log-dir/dag_1463533081218_0001_1-tez-dag.pb.txt
2016-05-18 00:58:01,386 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533081218_0001_1
2016-05-18 00:58:01,386 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:01,386 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [4045B0F073F64505879CA3F11D1AC7F4/A00AB4BA69D1491693583B434F772596] every.every/(1/1) ...ULTS.-1_'field2'_'field2'
2016-05-18 00:58:01 Running Dag: dag_1463533081218_0001_1
2016-05-18 00:58:01,396 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081218_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533081218_0001_1, submitTime=1463533081379
2016-05-18 00:58:01,398 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/4045B0F073F64505879CA3F11D1AC7F4/A00AB4BA69D1491693583B434F772596/.tez/application_1463533081218_0001/tez-conf.pb" } size: 36913 timestamp: 1463533081000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:01,398 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608]
2016-05-18 00:58:01,398 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608]
2016-05-18 00:58:01,398 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:01,399 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:01,399 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081218_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533081218_0001_1, initTime=1463533081399
2016-05-18 00:58:01,400 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533081218_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:01,401 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:01,402 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ULTS.-1_'field2'_'field2', applicationId=application_1463533081218_0001, dagName=[4045B0F073F64505879CA3F11D1AC7F4/A00AB4BA69D1491693583B434F772596] every.every/(1/1) ...ULTS.-1_'field2'_'field2'
2016-05-18 00:58:01,403 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081218_0001_1][Event:DAG_STARTED]: dagID=dag_1463533081218_0001_1, startTime=1463533081402
2016-05-18 00:58:01,403 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533081218_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:01,403 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: DFFFF258B3DC4649BFD687DA7D641608 : {5E2D32DF942543438243DF016AD60A15={InputName=5E2D32DF942543438243DF016AD60A15}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:01,403 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 5E2D32DF942543438243DF016AD60A15, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:01,403 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608]
2016-05-18 00:58:01,404 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608]
2016-05-18 00:58:01,404 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608]
2016-05-18 00:58:01,404 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608]
2016-05-18 00:58:01,406 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:01,406 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 5E2D32DF942543438243DF016AD60A15 on vertex vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608]
2016-05-18 00:58:01,419 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 5E2D32DF942543438243DF016AD60A15 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:01,423 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:01,424 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:01,425 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:01,425 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:01,427 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 5E2D32DF942543438243DF016AD60A15 on vertex vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608]
2016-05-18 00:58:01,428 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:01,429 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {5E2D32DF942543438243DF016AD60A15=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:01,431 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608] parallelism set to 1
2016-05-18 00:58:01,431 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:01,432 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608], numAdditionalOutputs=1
2016-05-18 00:58:01,433 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=3CA841FC014A49F0A5773B50874A4988, vertex=vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:01,433 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533081218_0001, with dag id: dag_1463533081218_0001_1
2016-05-18 00:58:01,438 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for DFFFF258B3DC4649BFD687DA7D641608:3CA841FC014A49F0A5773B50874A4988 using oldmapred API
2016-05-18 00:58:01,438 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=3CA841FC014A49F0A5773B50874A4988, vertexName=DFFFF258B3DC4649BFD687DA7D641608, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:01,440 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081218_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DFFFF258B3DC4649BFD687DA7D641608, vertexId=vertex_1463533081218_0001_1_00, initRequestedTime=1463533081403, initedTime=1463533081432, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:01,442 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608] with distanceFromRoot: 0
2016-05-18 00:58:01,442 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:01,443 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081218_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533081218_0001_1_00, startRequestedTime=1463533081406, startedTime=1463533081442
2016-05-18 00:58:01,444 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081218_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533081218_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:01,444 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:01,445 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in DFFFF258B3DC4649BFD687DA7D641608
2016-05-18 00:58:01,446 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608]
2016-05-18 00:58:01,448 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081218_0001_1][Event:TASK_STARTED]: vertexName=DFFFF258B3DC4649BFD687DA7D641608, taskId=task_1463533081218_0001_1_00_000000, scheduledTime=1463533081447, launchTime=1463533081447
2016-05-18 00:58:01,450 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608]
2016-05-18 00:58:01,451 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:01,454 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533081218_0001_00_000001, launchTime=1463533081454
2016-05-18 00:58:01,455 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533081218_0001_00_000001
2016-05-18 00:58:01,456 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533081218_0001_1_00_000000_0] started. Is using containerId: [container_1463533081218_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:01,457 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081218_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DFFFF258B3DC4649BFD687DA7D641608, taskAttemptId=attempt_1463533081218_0001_1_00_000000_0, startTime=1463533081456, containerId=container_1463533081218_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:01,458 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533081218_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533081218_0001_1_00_000000_0
2016-05-18 00:58:01 Starting to run new task attempt: attempt_1463533081218_0001_1_00_000000_0
2016-05-18 00:58:01,459 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533081218_0001_1_00_000000_0
2016-05-18 00:58:01,460 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:01,460 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:01,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [4045B0F073F64505879CA3F11D1AC7F4/A00AB4BA69D1491693583B434F772596] every.every/(1/1) ...ULTS.-1_'field2'_'field2', VertexName: DFFFF258B3DC4649BFD687DA7D641608, VertexParallelism: 1, TaskAttemptID:attempt_1463533081218_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5E2D32DF942543438243DF016AD60A15, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=3CA841FC014A49F0A5773B50874A4988, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:01,461 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:01,462 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533081218_0001_1_00_000000_0
2016-05-18 00:58:01,470 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:01,476 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533081218_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:01,477 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533081218_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:01,477 INFO input.MRInput (MRInput.java:initialize(440)) - 5E2D32DF942543438243DF016AD60A15 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:01,478 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 5E2D32DF942543438243DF016AD60A15
2016-05-18 00:58:01,488 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DFFFF258B3DC4649BFD687DA7D641608, ordinal: 0
2016-05-18 00:58:01,488 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DFFFF258B3DC4649BFD687DA7D641608, mem on start (mb), free: 281, total: 397, max: 672
2016-05-18 00:58:01,488 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:01,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:01,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:01,497 INFO output.MROutput (MROutput.java:initialize(421)) - 3CA841FC014A49F0A5773B50874A4988: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:01,498 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:01,498 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[5E2D32DF942543438243DF016AD60A15:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [3CA841FC014A49F0A5773B50874A4988:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:01,499 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:01,499 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:01,499 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:01,499 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[5E2D32DF942543438243DF016AD60A15:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [3CA841FC014A49F0A5773B50874A4988:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:01,500 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:01,500 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 5E2D32DF942543438243DF016AD60A15
2016-05-18 00:58:01,500 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 5E2D32DF942543438243DF016AD60A15 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:01,501 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:01,501 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:01,501 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:01,501 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533081218_0001_1_00_000000_0
2016-05-18 00:58:01,505 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:01,506 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 5E2D32DF942543438243DF016AD60A15 initialized RecordReader from event
2016-05-18 00:58:01,510 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:01,511 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 5E2D32DF942543438243DF016AD60A15
2016-05-18 00:58:01,511 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field2'"], id: 3CA841FC014A49F0A5773B50874A4988
2016-05-18 00:58:01,512 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field2'"]
2016-05-18 00:58:01,512 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:01,513 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DFFFF258B3DC4649BFD687DA7D641608, all 1 inputs ready in: 00:00:00.001
2016-05-18 00:58:01,518 INFO output.MROutput (MROutput.java:close(526)) - 3CA841FC014A49F0A5773B50874A4988 closed
2016-05-18 00:58:01,519 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533081218_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:01,520 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533081218_0001_1_00_000000_0
2016-05-18 00:58:01,520 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DFFFF258B3DC4649BFD687DA7D641608, mem on close (mb), free: 278, total: 397, max: 672
2016-05-18 00:58:01,521 INFO output.MROutput (MROutput.java:close(526)) - 3CA841FC014A49F0A5773B50874A4988 closed
2016-05-18 00:58:01,521 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:01 Completed running task attempt: attempt_1463533081218_0001_1_00_000000_0
2016-05-18 00:58:01,523 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533081218_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:01,523 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533081218_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:01,524 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533081218_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533081513, Process_Duration=7, Process_End_Time=1463533081520, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:01,525 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533081218_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:01,526 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:01,527 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:01,527 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:01,527 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533081218_0001_00_000001
2016-05-18 00:58:01,526 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081218_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DFFFF258B3DC4649BFD687DA7D641608, taskAttemptId=attempt_1463533081218_0001_1_00_000000_0, creationTime=1463533081449, allocationTime=1463533081453, startTime=1463533081456, finishTime=1463533081526, timeTaken=70, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:01,529 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533081218_0001_1_00_000000_0
2016-05-18 00:58:01,530 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081218_0001_1][Event:TASK_FINISHED]: vertexName=DFFFF258B3DC4649BFD687DA7D641608, taskId=task_1463533081218_0001_1_00_000000, startTime=1463533081456, finishTime=1463533081529, timeTaken=73, status=SUCCEEDED, successfulAttemptID=attempt_1463533081218_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533081513, Process_Duration=7, Process_End_Time=1463533081520, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:01,531 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:01,531 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608]
2016-05-18 00:58:01,534 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081218_0001_1][Event:VERTEX_FINISHED]: vertexName=DFFFF258B3DC4649BFD687DA7D641608, vertexId=vertex_1463533081218_0001_1_00, initRequestedTime=1463533081403, initedTime=1463533081432, startRequestedTime=1463533081406, startedTime=1463533081442, finishTime=1463533081531, timeTaken=89, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533081513, Process_Duration=7, Process_End_Time=1463533081520, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533081456, firstTasksToStart=[ task_1463533081218_0001_1_00_000000 ], lastTaskFinishTime=1463533081526, lastTasksToFinish=[ task_1463533081218_0001_1_00_000000 ], minTaskDuration=70, maxTaskDuration=70, avgTaskDuration=70.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533081218_0001_1_00_000000 ], longestDurationTasks=[ task_1463533081218_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:01,534 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:01,534 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081218_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533081218_0001_00_000001, stoppedTime=1463533081534, exitStatus=0
2016-05-18 00:58:01,534 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:01,534 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:01,535 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533081218_0001_1
2016-05-18 00:58:01,535 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081218_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533081218_0001_1
2016-05-18 00:58:01,535 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533081218_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:01,535 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 3CA841FC014A49F0A5773B50874A4988 for vertex: vertex_1463533081218_0001_1_00 [DFFFF258B3DC4649BFD687DA7D641608], outputName: 3CA841FC014A49F0A5773B50874A4988
2016-05-18 00:58:01,535 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533081218_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:01,544 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533081218_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:01,544 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533081218_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:01,544 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533081218_0001_00_000001, Exiting
2016-05-18 00:58:01,544 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533081218_0001_00_000001
2016-05-18 00:58:01,545 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533081218_0001_00_000001
2016-05-18 00:58:01,547 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:3CA841FC014A49F0A5773B50874A4988 of vertex/vertexGroup:DFFFF258B3DC4649BFD687DA7D641608 isVertexGroupOutput:false
2016-05-18 00:58:01,547 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:01,548 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533081218_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:01,548 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533081218_0001_00_000001 completed successfully
2016-05-18 00:58:01,549 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533081218_0001_1, queueSize=0
2016-05-18 00:58:01,551 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081218_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533081218_0001_1, startTime=1463533081401, finishTime=1463533081547, timeTaken=146, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=400, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533081513, Process_Duration=7, Process_End_Time=1463533081520, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:01,551 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533081218_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:01,551 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533081218_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:01 Completed Dag: dag_1463533081218_0001_1
2016-05-18 00:58:01,552 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533081218_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:01,552 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533081218_0001_1_post
2016-05-18 00:58:01,552 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:01,552 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:01,552 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:01,552 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[4045B0F073F64505879CA3F11D1AC7F4/A00AB4BA69D1491693583B434F772596] every.every/(1/1) ...ULTS.-1_'field2'_'field2', with id=dag_1463533081218_0001_1
2016-05-18 00:58:01,552 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[4045B0F073F64505879CA3F11D1AC7F4/A00AB4BA69D1491693583B434F772596] every.every/(1/1) ...ULTS.-1_'field2'_'field2', with id=dag_1463533081218_0001_1
2016-05-18 00:58:01,560 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ULTS.-1_'field2'_'field2', applicationId=application_1463533081218_0001
2016-05-18 00:58:01,564 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:01,564 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:01,564 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:01,564 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:01,564 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:01,565 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:01,566 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:01,566 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:01,567 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:01,567 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:01,568 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:01,569 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:01,569 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:01,571 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:01,576 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/4045B0F073F64505879CA3F11D1AC7F4/A00AB4BA69D1491693583B434F772596/.tez/application_1463533081218_0001
2016-05-18 00:58:01,577 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field2'/_temporary
2016-05-18 00:58:01,578 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.386
2016-05-18 00:58:01,677 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_'field2'_'field2'
2016-05-18 00:58:01,716 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:01,717 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:01,799 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-05-18 00:58:01,815 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.047
2016-05-18 00:58:01,816 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:01,816 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:01,817 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:01,846 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:01,848 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field'
2016-05-18 00:58:01,871 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:01,872 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:01,873 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field'"]
2016-05-18 00:58:01,873 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:01,873 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:01,873 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:01,875 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...SULTS.-1_'field2'_'field'
2016-05-18 00:58:01,876 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:01,900 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:01,906 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:01,906 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:01,906 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:01,916 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8B0A0C54B29B41F3B2C24F77596A5D7B/DAC063A68A2B45C4B2A57C520D6BEB36/.tez/application_1463533081906_0001 doesn't exist and is created
2016-05-18 00:58:01,939 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:01,939 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:01,942 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8B0A0C54B29B41F3B2C24F77596A5D7B/DAC063A68A2B45C4B2A57C520D6BEB36/.tez/application_1463533081906_0001_wd
2016-05-18 00:58:01,955 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533081906_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:01,955 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:01,955 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:01,961 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:01,961 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:01,962 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:01,962 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:01,963 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:01,967 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:01,967 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:01,967 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:01,967 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:01,968 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:01,968 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:01,968 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:01,969 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:01,969 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:01,969 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:01,970 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:01,970 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:01,973 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:01,974 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:01,975 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:01,975 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8B0A0C54B29B41F3B2C24F77596A5D7B/DAC063A68A2B45C4B2A57C520D6BEB36/.tez/application_1463533081906_0001_wd/localmode-log-dir
2016-05-18 00:58:01,975 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8B0A0C54B29B41F3B2C24F77596A5D7B/DAC063A68A2B45C4B2A57C520D6BEB36/.tez/application_1463533081906_0001_wd/localmode-log-dir/history.txt.appattempt_1463533081906_0001_000000, maxErrors=10
2016-05-18 00:58:01,976 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8B0A0C54B29B41F3B2C24F77596A5D7B/DAC063A68A2B45C4B2A57C520D6BEB36/.tez/application_1463533081906_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:01,976 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533081906_0001_000000, appSubmitTime=1463533081954, launchTime=1463533081954
2016-05-18 00:58:01,987 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:01,991 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:54856
2016-05-18 00:58:01,992 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533081906
2016-05-18 00:58:01,995 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533081906_0001_000000, startTime=1463533081995
2016-05-18 00:58:01,995 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:02,040 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:02,040 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:02,040 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...SULTS.-1_'field2'_'field', applicationId=application_1463533081906_0001, dagName=[8B0A0C54B29B41F3B2C24F77596A5D7B/DAC063A68A2B45C4B2A57C520D6BEB36] every.every/(1/1) ...SULTS.-1_'field2'_'field'
2016-05-18 00:58:02,070 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [8B0A0C54B29B41F3B2C24F77596A5D7B/DAC063A68A2B45C4B2A57C520D6BEB36] every.every/(1/1) ...SULTS.-1_'field2'_'field'
2016-05-18 00:58:02,070 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:02,071 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533081906_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8B0A0C54B29B41F3B2C24F77596A5D7B/DAC063A68A2B45C4B2A57C520D6BEB36/.tez/application_1463533081906_0001_wd/localmode-log-dir/dag_1463533081906_0001_1.dot
2016-05-18 00:58:02,072 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8B0A0C54B29B41F3B2C24F77596A5D7B/DAC063A68A2B45C4B2A57C520D6BEB36/.tez/application_1463533081906_0001_wd/localmode-log-dir/dag_1463533081906_0001_1-tez-dag.pb.txt
2016-05-18 00:58:02,078 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533081906_0001_1
2016-05-18 00:58:02,078 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:02,079 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [8B0A0C54B29B41F3B2C24F77596A5D7B/DAC063A68A2B45C4B2A57C520D6BEB36] every.every/(1/1) ...SULTS.-1_'field2'_'field'
2016-05-18 00:58:02 Running Dag: dag_1463533081906_0001_1
2016-05-18 00:58:02,089 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081906_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533081906_0001_1, submitTime=1463533082070
2016-05-18 00:58:02,091 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8B0A0C54B29B41F3B2C24F77596A5D7B/DAC063A68A2B45C4B2A57C520D6BEB36/.tez/application_1463533081906_0001/tez-conf.pb" } size: 36913 timestamp: 1463533081000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:02,091 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418]
2016-05-18 00:58:02,091 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418]
2016-05-18 00:58:02,092 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:02,093 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:02,094 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081906_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533081906_0001_1, initTime=1463533082093
2016-05-18 00:58:02,095 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533081906_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:02,095 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:02,096 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081906_0001_1][Event:DAG_STARTED]: dagID=dag_1463533081906_0001_1, startTime=1463533082096
2016-05-18 00:58:02,096 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533081906_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:02,096 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: FA1126C32EE048C89BD3829D38237418 : {9E91220E444D42DEBD85D2AE63684D08={InputName=9E91220E444D42DEBD85D2AE63684D08}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:02,096 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 9E91220E444D42DEBD85D2AE63684D08, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:02,096 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418]
2016-05-18 00:58:02,097 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418]
2016-05-18 00:58:02,097 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418]
2016-05-18 00:58:02,097 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...SULTS.-1_'field2'_'field', applicationId=application_1463533081906_0001, dagName=[8B0A0C54B29B41F3B2C24F77596A5D7B/DAC063A68A2B45C4B2A57C520D6BEB36] every.every/(1/1) ...SULTS.-1_'field2'_'field'
2016-05-18 00:58:02,097 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418]
2016-05-18 00:58:02,098 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:02,098 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 9E91220E444D42DEBD85D2AE63684D08 on vertex vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418]
2016-05-18 00:58:02,101 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 9E91220E444D42DEBD85D2AE63684D08 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:02,105 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:02,105 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:02,105 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:02,106 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:02,106 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 9E91220E444D42DEBD85D2AE63684D08 on vertex vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418]
2016-05-18 00:58:02,107 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:02,108 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {9E91220E444D42DEBD85D2AE63684D08=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:02,109 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418] parallelism set to 1
2016-05-18 00:58:02,110 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:02,111 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418], numAdditionalOutputs=1
2016-05-18 00:58:02,111 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=3873A0951C3F485A841A52C714B6D688, vertex=vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:02,114 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for FA1126C32EE048C89BD3829D38237418:3873A0951C3F485A841A52C714B6D688 using oldmapred API
2016-05-18 00:58:02,114 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=3873A0951C3F485A841A52C714B6D688, vertexName=FA1126C32EE048C89BD3829D38237418, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:02,118 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081906_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FA1126C32EE048C89BD3829D38237418, vertexId=vertex_1463533081906_0001_1_00, initRequestedTime=1463533082096, initedTime=1463533082111, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:02,118 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418] with distanceFromRoot: 0
2016-05-18 00:58:02,118 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:02,118 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081906_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533081906_0001_1_00, startRequestedTime=1463533082098, startedTime=1463533082118
2016-05-18 00:58:02,119 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081906_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533081906_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:02,119 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:02,120 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in FA1126C32EE048C89BD3829D38237418
2016-05-18 00:58:02,120 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418]
2016-05-18 00:58:02,121 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081906_0001_1][Event:TASK_STARTED]: vertexName=FA1126C32EE048C89BD3829D38237418, taskId=task_1463533081906_0001_1_00_000000, scheduledTime=1463533082121, launchTime=1463533082121
2016-05-18 00:58:02,122 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418]
2016-05-18 00:58:02,123 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:02,126 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533081906_0001_00_000001, launchTime=1463533082125
2016-05-18 00:58:02,126 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533081906_0001_00_000001
2016-05-18 00:58:02,127 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533081906_0001_1_00_000000_0] started. Is using containerId: [container_1463533081906_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:02,128 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081906_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FA1126C32EE048C89BD3829D38237418, taskAttemptId=attempt_1463533081906_0001_1_00_000000_0, startTime=1463533082127, containerId=container_1463533081906_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:02,129 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533081906_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533081906_0001_1_00_000000_0
2016-05-18 00:58:02,129 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533081906_0001, with dag id: dag_1463533081906_0001_1
2016-05-18 00:58:02 Starting to run new task attempt: attempt_1463533081906_0001_1_00_000000_0
2016-05-18 00:58:02,132 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533081906_0001_1_00_000000_0
2016-05-18 00:58:02,132 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:02,132 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:02,135 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8B0A0C54B29B41F3B2C24F77596A5D7B/DAC063A68A2B45C4B2A57C520D6BEB36] every.every/(1/1) ...SULTS.-1_'field2'_'field', VertexName: FA1126C32EE048C89BD3829D38237418, VertexParallelism: 1, TaskAttemptID:attempt_1463533081906_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9E91220E444D42DEBD85D2AE63684D08, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=3873A0951C3F485A841A52C714B6D688, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:02,136 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:02,138 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533081906_0001_1_00_000000_0
2016-05-18 00:58:02,141 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533081906_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:02,141 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533081906_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:02,145 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:02,146 INFO input.MRInput (MRInput.java:initialize(440)) - 9E91220E444D42DEBD85D2AE63684D08 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:02,149 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 9E91220E444D42DEBD85D2AE63684D08
2016-05-18 00:58:02,157 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FA1126C32EE048C89BD3829D38237418, ordinal: 0
2016-05-18 00:58:02,157 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FA1126C32EE048C89BD3829D38237418, mem on start (mb), free: 175, total: 397, max: 672
2016-05-18 00:58:02,159 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:02,159 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:02,159 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:02,160 INFO output.MROutput (MROutput.java:initialize(421)) - 3873A0951C3F485A841A52C714B6D688: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:02,161 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:02,161 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[9E91220E444D42DEBD85D2AE63684D08:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [3873A0951C3F485A841A52C714B6D688:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:02,162 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:02,162 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:02,163 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:02,163 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[9E91220E444D42DEBD85D2AE63684D08:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [3873A0951C3F485A841A52C714B6D688:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:02,164 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:02,164 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 9E91220E444D42DEBD85D2AE63684D08 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:02,165 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:02,165 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:02,164 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 9E91220E444D42DEBD85D2AE63684D08
2016-05-18 00:58:02,166 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:02,167 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533081906_0001_1_00_000000_0
2016-05-18 00:58:02,168 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:02,169 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 9E91220E444D42DEBD85D2AE63684D08 initialized RecordReader from event
2016-05-18 00:58:02,172 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:02,173 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 9E91220E444D42DEBD85D2AE63684D08
2016-05-18 00:58:02,173 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field'"], id: 3873A0951C3F485A841A52C714B6D688
2016-05-18 00:58:02,174 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field'"]
2016-05-18 00:58:02,174 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:02,175 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FA1126C32EE048C89BD3829D38237418, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:02,179 INFO output.MROutput (MROutput.java:close(526)) - 3873A0951C3F485A841A52C714B6D688 closed
2016-05-18 00:58:02,180 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533081906_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:02,181 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533081906_0001_1_00_000000_0
2016-05-18 00:58:02,182 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FA1126C32EE048C89BD3829D38237418, mem on close (mb), free: 173, total: 397, max: 672
2016-05-18 00:58:02,182 INFO output.MROutput (MROutput.java:close(526)) - 3873A0951C3F485A841A52C714B6D688 closed
2016-05-18 00:58:02,182 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:02 Completed running task attempt: attempt_1463533081906_0001_1_00_000000_0
2016-05-18 00:58:02,183 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533081906_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:02,184 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533081906_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:02,184 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533081906_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533082175, Process_Duration=6, Process_End_Time=1463533082181, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:02,186 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533081906_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:02,186 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081906_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FA1126C32EE048C89BD3829D38237418, taskAttemptId=attempt_1463533081906_0001_1_00_000000_0, creationTime=1463533082122, allocationTime=1463533082124, startTime=1463533082127, finishTime=1463533082186, timeTaken=59, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:02,186 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:02,187 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533081906_0001_1_00_000000_0
2016-05-18 00:58:02,187 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:02,187 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:02,188 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533081906_0001_00_000001
2016-05-18 00:58:02,189 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081906_0001_1][Event:TASK_FINISHED]: vertexName=FA1126C32EE048C89BD3829D38237418, taskId=task_1463533081906_0001_1_00_000000, startTime=1463533082127, finishTime=1463533082187, timeTaken=60, status=SUCCEEDED, successfulAttemptID=attempt_1463533081906_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533082175, Process_Duration=6, Process_End_Time=1463533082181, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:02,189 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:02,189 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418]
2016-05-18 00:58:02,191 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081906_0001_1][Event:VERTEX_FINISHED]: vertexName=FA1126C32EE048C89BD3829D38237418, vertexId=vertex_1463533081906_0001_1_00, initRequestedTime=1463533082096, initedTime=1463533082111, startRequestedTime=1463533082098, startedTime=1463533082118, finishTime=1463533082189, timeTaken=71, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533082175, Process_Duration=6, Process_End_Time=1463533082181, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533082127, firstTasksToStart=[ task_1463533081906_0001_1_00_000000 ], lastTaskFinishTime=1463533082186, lastTasksToFinish=[ task_1463533081906_0001_1_00_000000 ], minTaskDuration=59, maxTaskDuration=59, avgTaskDuration=59.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533081906_0001_1_00_000000 ], longestDurationTasks=[ task_1463533081906_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:02,192 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:02,192 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081906_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533081906_0001_00_000001, stoppedTime=1463533082192, exitStatus=0
2016-05-18 00:58:02,192 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:02,192 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:02,193 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533081906_0001_1
2016-05-18 00:58:02,194 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081906_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533081906_0001_1
2016-05-18 00:58:02,194 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 3873A0951C3F485A841A52C714B6D688 for vertex: vertex_1463533081906_0001_1_00 [FA1126C32EE048C89BD3829D38237418], outputName: 3873A0951C3F485A841A52C714B6D688
2016-05-18 00:58:02,199 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533081906_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:02,199 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533081906_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:02,199 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533081906_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:02,203 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533081906_0001_00_000001, Exiting
2016-05-18 00:58:02,203 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533081906_0001_00_000001
2016-05-18 00:58:02,204 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533081906_0001_00_000001
2016-05-18 00:58:02,204 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533081906_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:02,210 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:3873A0951C3F485A841A52C714B6D688 of vertex/vertexGroup:FA1126C32EE048C89BD3829D38237418 isVertexGroupOutput:false
2016-05-18 00:58:02,210 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:02,210 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533081906_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:02,211 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533081906_0001_00_000001 completed successfully
2016-05-18 00:58:02,212 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533081906_0001_1, queueSize=0
2016-05-18 00:58:02,214 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533081906_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533081906_0001_1, startTime=1463533082095, finishTime=1463533082210, timeTaken=115, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=210, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533082175, Process_Duration=6, Process_End_Time=1463533082181, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:02,214 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533081906_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:02,214 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533081906_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:02 Completed Dag: dag_1463533081906_0001_1
2016-05-18 00:58:02,215 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533081906_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:02,216 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533081906_0001_1_post
2016-05-18 00:58:02,216 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:02,216 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:02,216 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:02,217 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[8B0A0C54B29B41F3B2C24F77596A5D7B/DAC063A68A2B45C4B2A57C520D6BEB36] every.every/(1/1) ...SULTS.-1_'field2'_'field', with id=dag_1463533081906_0001_1
2016-05-18 00:58:02,217 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[8B0A0C54B29B41F3B2C24F77596A5D7B/DAC063A68A2B45C4B2A57C520D6BEB36] every.every/(1/1) ...SULTS.-1_'field2'_'field', with id=dag_1463533081906_0001_1
2016-05-18 00:58:02,224 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...SULTS.-1_'field2'_'field', applicationId=application_1463533081906_0001
2016-05-18 00:58:02,230 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:02,230 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:02,230 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:02,230 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:02,230 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:02,231 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:02,231 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:02,233 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:02,234 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:02,234 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:02,234 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:02,235 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:02,235 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:02,236 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:02,240 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8B0A0C54B29B41F3B2C24F77596A5D7B/DAC063A68A2B45C4B2A57C520D6BEB36/.tez/application_1463533081906_0001
2016-05-18 00:58:02,241 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field'/_temporary
2016-05-18 00:58:02,242 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.370
2016-05-18 00:58:02,356 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_'field2'_'field'
2016-05-18 00:58:02,438 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:02,439 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:02,495 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-05-18 00:58:02,498 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-05-18 00:58:02,498 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:02,499 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:02,499 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:02,518 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:02,519 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_RESULTS
2016-05-18 00:58:02,533 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:02,534 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:02,534 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_RESULTS"]
2016-05-18 00:58:02,534 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:02,534 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:02,534 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:02,535 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...SULTS.-1_'field2'_RESULTS
2016-05-18 00:58:02,537 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:02,554 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:02,558 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:02,559 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:02,559 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:02,569 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/66526747B96446A1A0AE05E980B4F507/DE834BF4281F479692CBB61BF7A8A061/.tez/application_1463533082559_0001 doesn't exist and is created
2016-05-18 00:58:02,588 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:02,589 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:02,590 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/66526747B96446A1A0AE05E980B4F507/DE834BF4281F479692CBB61BF7A8A061/.tez/application_1463533082559_0001_wd
2016-05-18 00:58:02,603 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533082559_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:02,603 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:02,603 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:02,606 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:02,606 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:02,606 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:02,606 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:02,608 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:02,609 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:02,609 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:02,610 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:02,610 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:02,610 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:02,610 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:02,611 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:02,611 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:02,611 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:02,612 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:02,612 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:02,612 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:02,614 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:02,615 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:02,616 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:02,616 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/66526747B96446A1A0AE05E980B4F507/DE834BF4281F479692CBB61BF7A8A061/.tez/application_1463533082559_0001_wd/localmode-log-dir
2016-05-18 00:58:02,617 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/66526747B96446A1A0AE05E980B4F507/DE834BF4281F479692CBB61BF7A8A061/.tez/application_1463533082559_0001_wd/localmode-log-dir/history.txt.appattempt_1463533082559_0001_000000, maxErrors=10
2016-05-18 00:58:02,617 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/66526747B96446A1A0AE05E980B4F507/DE834BF4281F479692CBB61BF7A8A061/.tez/application_1463533082559_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:02,618 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533082559_0001_000000, appSubmitTime=1463533082602, launchTime=1463533082602
2016-05-18 00:58:02,622 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:02,626 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:40121
2016-05-18 00:58:02,627 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533082559
2016-05-18 00:58:02,629 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533082559_0001_000000, startTime=1463533082629
2016-05-18 00:58:02,629 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:02,689 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:02,689 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:02,689 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...SULTS.-1_'field2'_RESULTS, applicationId=application_1463533082559_0001, dagName=[66526747B96446A1A0AE05E980B4F507/DE834BF4281F479692CBB61BF7A8A061] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS
2016-05-18 00:58:02,709 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [66526747B96446A1A0AE05E980B4F507/DE834BF4281F479692CBB61BF7A8A061] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS
2016-05-18 00:58:02,709 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:02,710 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533082559_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/66526747B96446A1A0AE05E980B4F507/DE834BF4281F479692CBB61BF7A8A061/.tez/application_1463533082559_0001_wd/localmode-log-dir/dag_1463533082559_0001_1.dot
2016-05-18 00:58:02,711 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/66526747B96446A1A0AE05E980B4F507/DE834BF4281F479692CBB61BF7A8A061/.tez/application_1463533082559_0001_wd/localmode-log-dir/dag_1463533082559_0001_1-tez-dag.pb.txt
2016-05-18 00:58:02,717 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533082559_0001_1
2016-05-18 00:58:02,717 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:02,717 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [66526747B96446A1A0AE05E980B4F507/DE834BF4281F479692CBB61BF7A8A061] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS
2016-05-18 00:58:02 Running Dag: dag_1463533082559_0001_1
2016-05-18 00:58:02,726 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533082559_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533082559_0001_1, submitTime=1463533082709
2016-05-18 00:58:02,728 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/66526747B96446A1A0AE05E980B4F507/DE834BF4281F479692CBB61BF7A8A061/.tez/application_1463533082559_0001/tez-conf.pb" } size: 36913 timestamp: 1463533082000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:02,728 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A]
2016-05-18 00:58:02,729 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A]
2016-05-18 00:58:02,729 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:02,729 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:02,730 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533082559_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533082559_0001_1, initTime=1463533082730
2016-05-18 00:58:02,730 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533082559_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:02,731 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:02,731 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533082559_0001_1][Event:DAG_STARTED]: dagID=dag_1463533082559_0001_1, startTime=1463533082731
2016-05-18 00:58:02,731 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533082559_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:02,732 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: A07318E8E12547BD9969392F0B3BDF2A : {4CBF2CAAA8CD44909FBA88EC037545A3={InputName=4CBF2CAAA8CD44909FBA88EC037545A3}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:02,732 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 4CBF2CAAA8CD44909FBA88EC037545A3, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:02,732 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A]
2016-05-18 00:58:02,732 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...SULTS.-1_'field2'_RESULTS, applicationId=application_1463533082559_0001, dagName=[66526747B96446A1A0AE05E980B4F507/DE834BF4281F479692CBB61BF7A8A061] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS
2016-05-18 00:58:02,732 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A]
2016-05-18 00:58:02,732 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A]
2016-05-18 00:58:02,732 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A]
2016-05-18 00:58:02,733 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:02,733 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 4CBF2CAAA8CD44909FBA88EC037545A3 on vertex vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A]
2016-05-18 00:58:02,735 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 4CBF2CAAA8CD44909FBA88EC037545A3 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:02,737 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:02,737 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:02,737 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:02,737 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:02,738 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 4CBF2CAAA8CD44909FBA88EC037545A3 on vertex vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A]
2016-05-18 00:58:02,739 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:02,745 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {4CBF2CAAA8CD44909FBA88EC037545A3=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:02,746 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A] parallelism set to 1
2016-05-18 00:58:02,746 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:02,748 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A], numAdditionalOutputs=1
2016-05-18 00:58:02,748 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=9026B6B028264B0A93D12DD12385DC70, vertex=vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:02,750 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for A07318E8E12547BD9969392F0B3BDF2A:9026B6B028264B0A93D12DD12385DC70 using oldmapred API
2016-05-18 00:58:02,750 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=9026B6B028264B0A93D12DD12385DC70, vertexName=A07318E8E12547BD9969392F0B3BDF2A, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:02,751 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533082559_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A07318E8E12547BD9969392F0B3BDF2A, vertexId=vertex_1463533082559_0001_1_00, initRequestedTime=1463533082732, initedTime=1463533082748, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:02,752 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A] with distanceFromRoot: 0
2016-05-18 00:58:02,752 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:02,753 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533082559_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533082559_0001_1_00, startRequestedTime=1463533082733, startedTime=1463533082753
2016-05-18 00:58:02,754 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533082559_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533082559_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:02,754 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:02,754 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in A07318E8E12547BD9969392F0B3BDF2A
2016-05-18 00:58:02,754 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A]
2016-05-18 00:58:02,755 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533082559_0001_1][Event:TASK_STARTED]: vertexName=A07318E8E12547BD9969392F0B3BDF2A, taskId=task_1463533082559_0001_1_00_000000, scheduledTime=1463533082755, launchTime=1463533082755
2016-05-18 00:58:02,756 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A]
2016-05-18 00:58:02,757 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:02,759 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533082559_0001_00_000001, launchTime=1463533082759
2016-05-18 00:58:02,759 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533082559_0001_00_000001
2016-05-18 00:58:02,760 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533082559_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533082559_0001_1_00_000000_0
2016-05-18 00:58:02,760 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533082559_0001_1_00_000000_0] started. Is using containerId: [container_1463533082559_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:02,761 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533082559_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A07318E8E12547BD9969392F0B3BDF2A, taskAttemptId=attempt_1463533082559_0001_1_00_000000_0, startTime=1463533082760, containerId=container_1463533082559_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:02 Starting to run new task attempt: attempt_1463533082559_0001_1_00_000000_0
2016-05-18 00:58:02,762 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533082559_0001_1_00_000000_0
2016-05-18 00:58:02,762 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:02,762 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:02,762 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [66526747B96446A1A0AE05E980B4F507/DE834BF4281F479692CBB61BF7A8A061] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS, VertexName: A07318E8E12547BD9969392F0B3BDF2A, VertexParallelism: 1, TaskAttemptID:attempt_1463533082559_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4CBF2CAAA8CD44909FBA88EC037545A3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=9026B6B028264B0A93D12DD12385DC70, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:02,763 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:02,763 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533082559_0001, with dag id: dag_1463533082559_0001_1
2016-05-18 00:58:02,764 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533082559_0001_1_00_000000_0
2016-05-18 00:58:02,766 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533082559_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:02,766 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533082559_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:02,773 INFO input.MRInput (MRInput.java:initialize(440)) - 4CBF2CAAA8CD44909FBA88EC037545A3 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:02,773 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 4CBF2CAAA8CD44909FBA88EC037545A3
2016-05-18 00:58:02,773 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:02,783 INFO output.MROutput (MROutput.java:initialize(421)) - 9026B6B028264B0A93D12DD12385DC70: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:02,789 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A07318E8E12547BD9969392F0B3BDF2A, ordinal: 0
2016-05-18 00:58:02,789 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A07318E8E12547BD9969392F0B3BDF2A, mem on start (mb), free: 300, total: 397, max: 672
2016-05-18 00:58:02,789 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:02,790 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:02,790 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:02,790 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:02,790 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[4CBF2CAAA8CD44909FBA88EC037545A3:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [9026B6B028264B0A93D12DD12385DC70:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:02,791 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:02,791 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:02,792 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:02,792 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[4CBF2CAAA8CD44909FBA88EC037545A3:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [9026B6B028264B0A93D12DD12385DC70:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:02,792 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:02,792 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 4CBF2CAAA8CD44909FBA88EC037545A3
2016-05-18 00:58:02,793 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 4CBF2CAAA8CD44909FBA88EC037545A3 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:02,793 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:02,793 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:02,793 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:02,794 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533082559_0001_1_00_000000_0
2016-05-18 00:58:02,796 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:02,796 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 4CBF2CAAA8CD44909FBA88EC037545A3 initialized RecordReader from event
2016-05-18 00:58:02,797 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:02,798 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 4CBF2CAAA8CD44909FBA88EC037545A3
2016-05-18 00:58:02,799 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_RESULTS"], id: 9026B6B028264B0A93D12DD12385DC70
2016-05-18 00:58:02,799 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_RESULTS"]
2016-05-18 00:58:02,800 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:02,800 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A07318E8E12547BD9969392F0B3BDF2A, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:02,803 INFO output.MROutput (MROutput.java:close(526)) - 9026B6B028264B0A93D12DD12385DC70 closed
2016-05-18 00:58:02,804 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533082559_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:02,805 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533082559_0001_1_00_000000_0
2016-05-18 00:58:02,805 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A07318E8E12547BD9969392F0B3BDF2A, mem on close (mb), free: 298, total: 397, max: 672
2016-05-18 00:58:02,806 INFO output.MROutput (MROutput.java:close(526)) - 9026B6B028264B0A93D12DD12385DC70 closed
2016-05-18 00:58:02,806 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:02 Completed running task attempt: attempt_1463533082559_0001_1_00_000000_0
2016-05-18 00:58:02,807 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533082559_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:02,807 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533082559_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:02,807 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533082559_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533082800, Process_Duration=5, Process_End_Time=1463533082805, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:02,809 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533082559_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:02,809 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:02,809 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:02,809 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533082559_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A07318E8E12547BD9969392F0B3BDF2A, taskAttemptId=attempt_1463533082559_0001_1_00_000000_0, creationTime=1463533082756, allocationTime=1463533082758, startTime=1463533082760, finishTime=1463533082809, timeTaken=49, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:02,811 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533082559_0001_1_00_000000_0
2016-05-18 00:58:02,811 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:02,811 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533082559_0001_00_000001
2016-05-18 00:58:02,811 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533082559_0001_1][Event:TASK_FINISHED]: vertexName=A07318E8E12547BD9969392F0B3BDF2A, taskId=task_1463533082559_0001_1_00_000000, startTime=1463533082760, finishTime=1463533082811, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1463533082559_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533082800, Process_Duration=5, Process_End_Time=1463533082805, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:02,812 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:02,813 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A]
2016-05-18 00:58:02,815 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533082559_0001_1][Event:VERTEX_FINISHED]: vertexName=A07318E8E12547BD9969392F0B3BDF2A, vertexId=vertex_1463533082559_0001_1_00, initRequestedTime=1463533082732, initedTime=1463533082748, startRequestedTime=1463533082733, startedTime=1463533082753, finishTime=1463533082813, timeTaken=60, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533082800, Process_Duration=5, Process_End_Time=1463533082805, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533082760, firstTasksToStart=[ task_1463533082559_0001_1_00_000000 ], lastTaskFinishTime=1463533082809, lastTasksToFinish=[ task_1463533082559_0001_1_00_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533082559_0001_1_00_000000 ], longestDurationTasks=[ task_1463533082559_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:02,815 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:02,815 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533082559_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533082559_0001_00_000001, stoppedTime=1463533082815, exitStatus=0
2016-05-18 00:58:02,815 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:02,816 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:02,816 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533082559_0001_1
2016-05-18 00:58:02,816 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533082559_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533082559_0001_1
2016-05-18 00:58:02,816 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 9026B6B028264B0A93D12DD12385DC70 for vertex: vertex_1463533082559_0001_1_00 [A07318E8E12547BD9969392F0B3BDF2A], outputName: 9026B6B028264B0A93D12DD12385DC70
2016-05-18 00:58:02,816 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533082559_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:02,817 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533082559_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:02,823 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:9026B6B028264B0A93D12DD12385DC70 of vertex/vertexGroup:A07318E8E12547BD9969392F0B3BDF2A isVertexGroupOutput:false
2016-05-18 00:58:02,823 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:02,825 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533082559_0001_1, queueSize=0
2016-05-18 00:58:02,825 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533082559_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:02,825 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533082559_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:02,826 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533082559_0001_00_000001, Exiting
2016-05-18 00:58:02,826 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533082559_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533082559_0001_1, startTime=1463533082731, finishTime=1463533082823, timeTaken=92, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=160, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533082800, Process_Duration=5, Process_End_Time=1463533082805, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:02,826 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533082559_0001_00_000001
2016-05-18 00:58:02,826 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533082559_0001_00_000001
2016-05-18 00:58:02,826 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533082559_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:02,827 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533082559_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:02 Completed Dag: dag_1463533082559_0001_1
2016-05-18 00:58:02,828 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533082559_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:02,831 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533082559_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:02,832 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533082559_0001_1_post
2016-05-18 00:58:02,832 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:02,832 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:02,832 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:02,832 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[66526747B96446A1A0AE05E980B4F507/DE834BF4281F479692CBB61BF7A8A061] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS, with id=dag_1463533082559_0001_1
2016-05-18 00:58:02,833 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[66526747B96446A1A0AE05E980B4F507/DE834BF4281F479692CBB61BF7A8A061] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS, with id=dag_1463533082559_0001_1
2016-05-18 00:58:02,832 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533082559_0001_00_000001 completed successfully
2016-05-18 00:58:02,839 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...SULTS.-1_'field2'_RESULTS, applicationId=application_1463533082559_0001
2016-05-18 00:58:02,842 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:02,842 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:02,842 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:02,842 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:02,842 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:02,842 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:02,843 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:02,844 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:02,844 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:02,844 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:02,844 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:02,845 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:02,845 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:02,846 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:02,850 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/66526747B96446A1A0AE05E980B4F507/DE834BF4281F479692CBB61BF7A8A061/.tez/application_1463533082559_0001
2016-05-18 00:58:02,850 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_RESULTS/_temporary
2016-05-18 00:58:02,851 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.317
2016-05-18 00:58:02,949 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_'field2'_RESULTS
2016-05-18 00:58:02,979 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:02,980 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:03,029 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-18 00:58:03,052 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.041
2016-05-18 00:58:03,053 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:03,053 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:03,053 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:03,071 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:03,071 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_ALL
2016-05-18 00:58:03,084 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:03,085 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:03,085 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_ALL"]
2016-05-18 00:58:03,085 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:03,085 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:03,085 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:03,087 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...'_RESULTS.-1_'field2'_ALL
2016-05-18 00:58:03,087 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:03,105 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:03,116 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:03,116 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:03,117 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:03,126 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/64EE5D7BAEB041C481DAE18BE23FC0DF/E6ECA00F460D4F8B9CC4960044AC41F5/.tez/application_1463533083116_0001 doesn't exist and is created
2016-05-18 00:58:03,140 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:03,141 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:03,142 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/64EE5D7BAEB041C481DAE18BE23FC0DF/E6ECA00F460D4F8B9CC4960044AC41F5/.tez/application_1463533083116_0001_wd
2016-05-18 00:58:03,151 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533083116_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:03,152 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:03,152 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:03,153 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:03,153 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:03,153 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:03,153 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:03,154 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:03,157 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:03,157 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:03,157 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:03,157 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:03,157 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:03,157 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:03,158 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:03,158 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:03,158 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:03,158 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:03,158 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:03,158 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:03,161 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:03,163 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:03,163 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:03,163 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/64EE5D7BAEB041C481DAE18BE23FC0DF/E6ECA00F460D4F8B9CC4960044AC41F5/.tez/application_1463533083116_0001_wd/localmode-log-dir
2016-05-18 00:58:03,163 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/64EE5D7BAEB041C481DAE18BE23FC0DF/E6ECA00F460D4F8B9CC4960044AC41F5/.tez/application_1463533083116_0001_wd/localmode-log-dir/history.txt.appattempt_1463533083116_0001_000000, maxErrors=10
2016-05-18 00:58:03,164 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/64EE5D7BAEB041C481DAE18BE23FC0DF/E6ECA00F460D4F8B9CC4960044AC41F5/.tez/application_1463533083116_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:03,164 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533083116_0001_000000, appSubmitTime=1463533083151, launchTime=1463533083151
2016-05-18 00:58:03,170 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:03,173 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:47260
2016-05-18 00:58:03,174 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533083116
2016-05-18 00:58:03,176 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533083116_0001_000000, startTime=1463533083176
2016-05-18 00:58:03,176 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:03,241 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:03,242 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:03,242 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...'_RESULTS.-1_'field2'_ALL, applicationId=application_1463533083116_0001, dagName=[64EE5D7BAEB041C481DAE18BE23FC0DF/E6ECA00F460D4F8B9CC4960044AC41F5] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL
2016-05-18 00:58:03,262 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [64EE5D7BAEB041C481DAE18BE23FC0DF/E6ECA00F460D4F8B9CC4960044AC41F5] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL
2016-05-18 00:58:03,263 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:03,264 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533083116_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/64EE5D7BAEB041C481DAE18BE23FC0DF/E6ECA00F460D4F8B9CC4960044AC41F5/.tez/application_1463533083116_0001_wd/localmode-log-dir/dag_1463533083116_0001_1.dot
2016-05-18 00:58:03,265 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/64EE5D7BAEB041C481DAE18BE23FC0DF/E6ECA00F460D4F8B9CC4960044AC41F5/.tez/application_1463533083116_0001_wd/localmode-log-dir/dag_1463533083116_0001_1-tez-dag.pb.txt
2016-05-18 00:58:03,270 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533083116_0001_1
2016-05-18 00:58:03,270 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:03,270 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [64EE5D7BAEB041C481DAE18BE23FC0DF/E6ECA00F460D4F8B9CC4960044AC41F5] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL
2016-05-18 00:58:03 Running Dag: dag_1463533083116_0001_1
2016-05-18 00:58:03,281 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083116_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533083116_0001_1, submitTime=1463533083262
2016-05-18 00:58:03,283 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/64EE5D7BAEB041C481DAE18BE23FC0DF/E6ECA00F460D4F8B9CC4960044AC41F5/.tez/application_1463533083116_0001/tez-conf.pb" } size: 36913 timestamp: 1463533083000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:03,283 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4]
2016-05-18 00:58:03,283 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4]
2016-05-18 00:58:03,284 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:03,285 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:03,286 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083116_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533083116_0001_1, initTime=1463533083285
2016-05-18 00:58:03,287 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533083116_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:03,287 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:03,287 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083116_0001_1][Event:DAG_STARTED]: dagID=dag_1463533083116_0001_1, startTime=1463533083287
2016-05-18 00:58:03,288 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533083116_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:03,288 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: F293B0FA8E6D4BDFB0801BAAA23232B4 : {C678078668BD4DFAAC17543C15CD0F47={InputName=C678078668BD4DFAAC17543C15CD0F47}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:03,288 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: C678078668BD4DFAAC17543C15CD0F47, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:03,288 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4]
2016-05-18 00:58:03,288 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4]
2016-05-18 00:58:03,288 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4]
2016-05-18 00:58:03,289 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...'_RESULTS.-1_'field2'_ALL, applicationId=application_1463533083116_0001, dagName=[64EE5D7BAEB041C481DAE18BE23FC0DF/E6ECA00F460D4F8B9CC4960044AC41F5] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL
2016-05-18 00:58:03,289 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4]
2016-05-18 00:58:03,289 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:03,290 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: C678078668BD4DFAAC17543C15CD0F47 on vertex vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4]
2016-05-18 00:58:03,292 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input C678078668BD4DFAAC17543C15CD0F47 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:03,296 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:03,296 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:03,297 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:03,297 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:03,298 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: C678078668BD4DFAAC17543C15CD0F47 on vertex vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4]
2016-05-18 00:58:03,299 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:03,299 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {C678078668BD4DFAAC17543C15CD0F47=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:03,300 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4] parallelism set to 1
2016-05-18 00:58:03,300 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:03,301 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4], numAdditionalOutputs=1
2016-05-18 00:58:03,301 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=2514097AEAC447F9869CB75C4BC94862, vertex=vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:03,304 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for F293B0FA8E6D4BDFB0801BAAA23232B4:2514097AEAC447F9869CB75C4BC94862 using oldmapred API
2016-05-18 00:58:03,304 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=2514097AEAC447F9869CB75C4BC94862, vertexName=F293B0FA8E6D4BDFB0801BAAA23232B4, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:03,305 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083116_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F293B0FA8E6D4BDFB0801BAAA23232B4, vertexId=vertex_1463533083116_0001_1_00, initRequestedTime=1463533083288, initedTime=1463533083301, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:03,306 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4] with distanceFromRoot: 0
2016-05-18 00:58:03,307 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:03,308 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083116_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533083116_0001_1_00, startRequestedTime=1463533083289, startedTime=1463533083307
2016-05-18 00:58:03,308 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083116_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533083116_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:03,309 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:03,309 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in F293B0FA8E6D4BDFB0801BAAA23232B4
2016-05-18 00:58:03,309 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4]
2016-05-18 00:58:03,310 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083116_0001_1][Event:TASK_STARTED]: vertexName=F293B0FA8E6D4BDFB0801BAAA23232B4, taskId=task_1463533083116_0001_1_00_000000, scheduledTime=1463533083310, launchTime=1463533083310
2016-05-18 00:58:03,311 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4]
2016-05-18 00:58:03,312 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:03,314 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533083116_0001_00_000001, launchTime=1463533083314
2016-05-18 00:58:03,315 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533083116_0001_00_000001
2016-05-18 00:58:03,316 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533083116_0001_1_00_000000_0] started. Is using containerId: [container_1463533083116_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:03,316 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083116_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F293B0FA8E6D4BDFB0801BAAA23232B4, taskAttemptId=attempt_1463533083116_0001_1_00_000000_0, startTime=1463533083316, containerId=container_1463533083116_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:03,317 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533083116_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533083116_0001_1_00_000000_0
2016-05-18 00:58:03 Starting to run new task attempt: attempt_1463533083116_0001_1_00_000000_0
2016-05-18 00:58:03,318 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533083116_0001_1_00_000000_0
2016-05-18 00:58:03,318 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:03,319 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:03,319 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [64EE5D7BAEB041C481DAE18BE23FC0DF/E6ECA00F460D4F8B9CC4960044AC41F5] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL, VertexName: F293B0FA8E6D4BDFB0801BAAA23232B4, VertexParallelism: 1, TaskAttemptID:attempt_1463533083116_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C678078668BD4DFAAC17543C15CD0F47, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=2514097AEAC447F9869CB75C4BC94862, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:03,320 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:03,320 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533083116_0001_1_00_000000_0
2016-05-18 00:58:03,325 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533083116_0001, with dag id: dag_1463533083116_0001_1
2016-05-18 00:58:03,331 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:03,332 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533083116_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:03,333 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533083116_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:03,336 INFO input.MRInput (MRInput.java:initialize(440)) - C678078668BD4DFAAC17543C15CD0F47 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:03,336 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: C678078668BD4DFAAC17543C15CD0F47
2016-05-18 00:58:03,341 INFO output.MROutput (MROutput.java:initialize(421)) - 2514097AEAC447F9869CB75C4BC94862: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:03,344 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F293B0FA8E6D4BDFB0801BAAA23232B4, ordinal: 0
2016-05-18 00:58:03,344 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F293B0FA8E6D4BDFB0801BAAA23232B4, mem on start (mb), free: 201, total: 397, max: 672
2016-05-18 00:58:03,344 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:03,344 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:03,344 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:03,344 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:03,345 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[2514097AEAC447F9869CB75C4BC94862:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [C678078668BD4DFAAC17543C15CD0F47:INPUT:0:org.apache.tez.mapreduce.input.MRInput]
2016-05-18 00:58:03,345 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:03,345 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:03,345 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:03,345 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[2514097AEAC447F9869CB75C4BC94862:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [C678078668BD4DFAAC17543C15CD0F47:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0]
2016-05-18 00:58:03,345 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:03,346 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: C678078668BD4DFAAC17543C15CD0F47 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:03,346 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:03,346 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:03,346 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: C678078668BD4DFAAC17543C15CD0F47
2016-05-18 00:58:03,346 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:03,346 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533083116_0001_1_00_000000_0
2016-05-18 00:58:03,350 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:03,351 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - C678078668BD4DFAAC17543C15CD0F47 initialized RecordReader from event
2016-05-18 00:58:03,355 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:03,356 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: C678078668BD4DFAAC17543C15CD0F47
2016-05-18 00:58:03,356 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_ALL"], id: 2514097AEAC447F9869CB75C4BC94862
2016-05-18 00:58:03,357 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_ALL"]
2016-05-18 00:58:03,357 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:03,358 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F293B0FA8E6D4BDFB0801BAAA23232B4, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:03,361 INFO output.MROutput (MROutput.java:close(526)) - 2514097AEAC447F9869CB75C4BC94862 closed
2016-05-18 00:58:03,362 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533083116_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:03,363 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533083116_0001_1_00_000000_0
2016-05-18 00:58:03,363 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F293B0FA8E6D4BDFB0801BAAA23232B4, mem on close (mb), free: 199, total: 397, max: 672
2016-05-18 00:58:03,363 INFO output.MROutput (MROutput.java:close(526)) - 2514097AEAC447F9869CB75C4BC94862 closed
2016-05-18 00:58:03,363 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:03 Completed running task attempt: attempt_1463533083116_0001_1_00_000000_0
2016-05-18 00:58:03,364 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533083116_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:03,364 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533083116_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:03,365 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533083116_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533083358, Process_Duration=5, Process_End_Time=1463533083363, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:03,366 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533083116_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:03,366 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:03,366 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:03,366 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:03,367 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083116_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F293B0FA8E6D4BDFB0801BAAA23232B4, taskAttemptId=attempt_1463533083116_0001_1_00_000000_0, creationTime=1463533083311, allocationTime=1463533083313, startTime=1463533083316, finishTime=1463533083367, timeTaken=51, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:03,368 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533083116_0001_1_00_000000_0
2016-05-18 00:58:03,368 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083116_0001_1][Event:TASK_FINISHED]: vertexName=F293B0FA8E6D4BDFB0801BAAA23232B4, taskId=task_1463533083116_0001_1_00_000000, startTime=1463533083316, finishTime=1463533083368, timeTaken=52, status=SUCCEEDED, successfulAttemptID=attempt_1463533083116_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533083358, Process_Duration=5, Process_End_Time=1463533083363, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:03,369 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:03,369 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4]
2016-05-18 00:58:03,367 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533083116_0001_00_000001
2016-05-18 00:58:03,370 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083116_0001_1][Event:VERTEX_FINISHED]: vertexName=F293B0FA8E6D4BDFB0801BAAA23232B4, vertexId=vertex_1463533083116_0001_1_00, initRequestedTime=1463533083288, initedTime=1463533083301, startRequestedTime=1463533083289, startedTime=1463533083307, finishTime=1463533083369, timeTaken=62, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533083358, Process_Duration=5, Process_End_Time=1463533083363, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533083316, firstTasksToStart=[ task_1463533083116_0001_1_00_000000 ], lastTaskFinishTime=1463533083367, lastTasksToFinish=[ task_1463533083116_0001_1_00_000000 ], minTaskDuration=51, maxTaskDuration=51, avgTaskDuration=51.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533083116_0001_1_00_000000 ], longestDurationTasks=[ task_1463533083116_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:03,371 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:03,371 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083116_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533083116_0001_00_000001, stoppedTime=1463533083371, exitStatus=0
2016-05-18 00:58:03,371 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:03,371 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:03,372 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533083116_0001_1
2016-05-18 00:58:03,372 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083116_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533083116_0001_1
2016-05-18 00:58:03,372 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 2514097AEAC447F9869CB75C4BC94862 for vertex: vertex_1463533083116_0001_1_00 [F293B0FA8E6D4BDFB0801BAAA23232B4], outputName: 2514097AEAC447F9869CB75C4BC94862
2016-05-18 00:58:03,372 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533083116_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:03,373 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533083116_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:03,373 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533083116_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:03,373 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533083116_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:03,374 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533083116_0001_00_000001, Exiting
2016-05-18 00:58:03,374 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533083116_0001_00_000001
2016-05-18 00:58:03,374 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533083116_0001_00_000001
2016-05-18 00:58:03,375 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533083116_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:03,375 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533083116_0001_00_000001 completed successfully
2016-05-18 00:58:03,383 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:2514097AEAC447F9869CB75C4BC94862 of vertex/vertexGroup:F293B0FA8E6D4BDFB0801BAAA23232B4 isVertexGroupOutput:false
2016-05-18 00:58:03,383 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:03,385 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533083116_0001_1, queueSize=0
2016-05-18 00:58:03,385 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083116_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533083116_0001_1, startTime=1463533083287, finishTime=1463533083383, timeTaken=96, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=180, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533083358, Process_Duration=5, Process_End_Time=1463533083363, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:03,385 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533083116_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:03,385 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533083116_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:03 Completed Dag: dag_1463533083116_0001_1
2016-05-18 00:58:03,386 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533083116_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:03,386 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533083116_0001_1_post
2016-05-18 00:58:03,386 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:03,386 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:03,386 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:03,386 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[64EE5D7BAEB041C481DAE18BE23FC0DF/E6ECA00F460D4F8B9CC4960044AC41F5] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL, with id=dag_1463533083116_0001_1
2016-05-18 00:58:03,387 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[64EE5D7BAEB041C481DAE18BE23FC0DF/E6ECA00F460D4F8B9CC4960044AC41F5] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL, with id=dag_1463533083116_0001_1
2016-05-18 00:58:03,395 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...'_RESULTS.-1_'field2'_ALL, applicationId=application_1463533083116_0001
2016-05-18 00:58:03,397 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:03,397 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:03,398 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:03,398 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:03,398 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:03,398 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:03,400 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:03,400 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:03,400 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:03,401 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:03,401 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:03,402 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:03,402 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:03,403 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:03,407 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/64EE5D7BAEB041C481DAE18BE23FC0DF/E6ECA00F460D4F8B9CC4960044AC41F5/.tez/application_1463533083116_0001
2016-05-18 00:58:03,408 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_ALL/_temporary
2016-05-18 00:58:03,408 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.323
2016-05-18 00:58:03,510 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_'field2'_ALL
2016-05-18 00:58:03,542 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:03,543 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:03,614 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-05-18 00:58:03,624 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.034
2016-05-18 00:58:03,625 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:03,625 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:03,626 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:03,647 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:03,647 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_-1
2016-05-18 00:58:03,660 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:03,661 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:03,661 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_-1"]
2016-05-18 00:58:03,661 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:03,662 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:03,662 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:03,664 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ld'_RESULTS.-1_UNKNOWN_-1
2016-05-18 00:58:03,664 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:03,682 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:03,687 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:03,688 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:03,688 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:03,698 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/6D1B6F8AA2EA404697C22374F0029574/F95E85D0E47C41199E90ECD2B0389EB4/.tez/application_1463533083687_0001 doesn't exist and is created
2016-05-18 00:58:03,714 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:03,715 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:03,716 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/6D1B6F8AA2EA404697C22374F0029574/F95E85D0E47C41199E90ECD2B0389EB4/.tez/application_1463533083687_0001_wd
2016-05-18 00:58:03,729 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533083687_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:03,729 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:03,729 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:03,731 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:03,731 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:03,731 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:03,732 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:03,734 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:03,735 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:03,735 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:03,736 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:03,736 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:03,736 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:03,737 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:03,737 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:03,737 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:03,737 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:03,737 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:03,737 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:03,737 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:03,740 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:03,741 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:03,741 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:03,742 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/6D1B6F8AA2EA404697C22374F0029574/F95E85D0E47C41199E90ECD2B0389EB4/.tez/application_1463533083687_0001_wd/localmode-log-dir
2016-05-18 00:58:03,742 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/6D1B6F8AA2EA404697C22374F0029574/F95E85D0E47C41199E90ECD2B0389EB4/.tez/application_1463533083687_0001_wd/localmode-log-dir/history.txt.appattempt_1463533083687_0001_000000, maxErrors=10
2016-05-18 00:58:03,742 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/6D1B6F8AA2EA404697C22374F0029574/F95E85D0E47C41199E90ECD2B0389EB4/.tez/application_1463533083687_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:03,743 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533083687_0001_000000, appSubmitTime=1463533083728, launchTime=1463533083728
2016-05-18 00:58:03,779 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:03,788 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:43100
2016-05-18 00:58:03,789 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533083687
2016-05-18 00:58:03,791 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533083687_0001_000000, startTime=1463533083791
2016-05-18 00:58:03,791 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:03,815 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:03,815 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:03,815 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1, applicationId=application_1463533083687_0001, dagName=[6D1B6F8AA2EA404697C22374F0029574/F95E85D0E47C41199E90ECD2B0389EB4] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1
2016-05-18 00:58:03,835 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [6D1B6F8AA2EA404697C22374F0029574/F95E85D0E47C41199E90ECD2B0389EB4] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1
2016-05-18 00:58:03,836 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:03,837 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533083687_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/6D1B6F8AA2EA404697C22374F0029574/F95E85D0E47C41199E90ECD2B0389EB4/.tez/application_1463533083687_0001_wd/localmode-log-dir/dag_1463533083687_0001_1.dot
2016-05-18 00:58:03,838 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/6D1B6F8AA2EA404697C22374F0029574/F95E85D0E47C41199E90ECD2B0389EB4/.tez/application_1463533083687_0001_wd/localmode-log-dir/dag_1463533083687_0001_1-tez-dag.pb.txt
2016-05-18 00:58:03,844 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533083687_0001_1
2016-05-18 00:58:03,844 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:03,844 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [6D1B6F8AA2EA404697C22374F0029574/F95E85D0E47C41199E90ECD2B0389EB4] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1
2016-05-18 00:58:03 Running Dag: dag_1463533083687_0001_1
2016-05-18 00:58:03,854 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083687_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533083687_0001_1, submitTime=1463533083836
2016-05-18 00:58:03,855 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/6D1B6F8AA2EA404697C22374F0029574/F95E85D0E47C41199E90ECD2B0389EB4/.tez/application_1463533083687_0001/tez-conf.pb" } size: 36913 timestamp: 1463533083000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:03,855 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9]
2016-05-18 00:58:03,856 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9]
2016-05-18 00:58:03,856 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:03,856 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:03,857 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083687_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533083687_0001_1, initTime=1463533083857
2016-05-18 00:58:03,857 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533083687_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:03,858 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:03,858 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083687_0001_1][Event:DAG_STARTED]: dagID=dag_1463533083687_0001_1, startTime=1463533083858
2016-05-18 00:58:03,859 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1, applicationId=application_1463533083687_0001, dagName=[6D1B6F8AA2EA404697C22374F0029574/F95E85D0E47C41199E90ECD2B0389EB4] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1
2016-05-18 00:58:03,859 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533083687_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:03,859 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: A02AE19A87E4469A8A051FBCF8C396F9 : {E2161B6282074C52987A63377982F976={InputName=E2161B6282074C52987A63377982F976}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:03,859 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: E2161B6282074C52987A63377982F976, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:03,859 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9]
2016-05-18 00:58:03,859 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9]
2016-05-18 00:58:03,860 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9]
2016-05-18 00:58:03,860 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9]
2016-05-18 00:58:03,861 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:03,861 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: E2161B6282074C52987A63377982F976 on vertex vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9]
2016-05-18 00:58:03,863 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input E2161B6282074C52987A63377982F976 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:03,864 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:03,865 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:03,865 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:03,865 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:03,867 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: E2161B6282074C52987A63377982F976 on vertex vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9]
2016-05-18 00:58:03,868 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:03,869 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {E2161B6282074C52987A63377982F976=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:03,869 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9] parallelism set to 1
2016-05-18 00:58:03,869 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:03,870 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9], numAdditionalOutputs=1
2016-05-18 00:58:03,870 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=81DF67B98E03441EB708F5D8A7622FBF, vertex=vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:03,872 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for A02AE19A87E4469A8A051FBCF8C396F9:81DF67B98E03441EB708F5D8A7622FBF using oldmapred API
2016-05-18 00:58:03,873 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=81DF67B98E03441EB708F5D8A7622FBF, vertexName=A02AE19A87E4469A8A051FBCF8C396F9, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:03,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083687_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A02AE19A87E4469A8A051FBCF8C396F9, vertexId=vertex_1463533083687_0001_1_00, initRequestedTime=1463533083859, initedTime=1463533083870, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:03,874 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9] with distanceFromRoot: 0
2016-05-18 00:58:03,875 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:03,875 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083687_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533083687_0001_1_00, startRequestedTime=1463533083861, startedTime=1463533083875
2016-05-18 00:58:03,875 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083687_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533083687_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:03,876 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:03,876 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in A02AE19A87E4469A8A051FBCF8C396F9
2016-05-18 00:58:03,876 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9]
2016-05-18 00:58:03,879 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083687_0001_1][Event:TASK_STARTED]: vertexName=A02AE19A87E4469A8A051FBCF8C396F9, taskId=task_1463533083687_0001_1_00_000000, scheduledTime=1463533083879, launchTime=1463533083879
2016-05-18 00:58:03,880 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9]
2016-05-18 00:58:03,881 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:03,883 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533083687_0001_00_000001, launchTime=1463533083883
2016-05-18 00:58:03,884 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533083687_0001_00_000001
2016-05-18 00:58:03,885 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533083687_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533083687_0001_1_00_000000_0
2016-05-18 00:58:03,886 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533083687_0001_1_00_000000_0] started. Is using containerId: [container_1463533083687_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:03 Starting to run new task attempt: attempt_1463533083687_0001_1_00_000000_0
2016-05-18 00:58:03,886 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083687_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A02AE19A87E4469A8A051FBCF8C396F9, taskAttemptId=attempt_1463533083687_0001_1_00_000000_0, startTime=1463533083885, containerId=container_1463533083687_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:03,886 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533083687_0001_1_00_000000_0
2016-05-18 00:58:03,886 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:03,886 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:03,887 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6D1B6F8AA2EA404697C22374F0029574/F95E85D0E47C41199E90ECD2B0389EB4] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1, VertexName: A02AE19A87E4469A8A051FBCF8C396F9, VertexParallelism: 1, TaskAttemptID:attempt_1463533083687_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E2161B6282074C52987A63377982F976, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=81DF67B98E03441EB708F5D8A7622FBF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:03,887 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:03,889 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533083687_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:03,889 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533083687_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:03,890 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533083687_0001, with dag id: dag_1463533083687_0001_1
2016-05-18 00:58:03,891 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533083687_0001_1_00_000000_0
2016-05-18 00:58:03,897 INFO input.MRInput (MRInput.java:initialize(440)) - E2161B6282074C52987A63377982F976 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:03,898 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: E2161B6282074C52987A63377982F976
2016-05-18 00:58:03,898 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:03,903 INFO output.MROutput (MROutput.java:initialize(421)) - 81DF67B98E03441EB708F5D8A7622FBF: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:03,909 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A02AE19A87E4469A8A051FBCF8C396F9, ordinal: 0
2016-05-18 00:58:03,909 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A02AE19A87E4469A8A051FBCF8C396F9, mem on start (mb), free: 286, total: 359, max: 672
2016-05-18 00:58:03,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:03,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:03,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:03,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:03,909 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[E2161B6282074C52987A63377982F976:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [81DF67B98E03441EB708F5D8A7622FBF:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:03,910 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:03,910 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:03,911 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:03,911 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[E2161B6282074C52987A63377982F976:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [81DF67B98E03441EB708F5D8A7622FBF:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:03,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:03,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: E2161B6282074C52987A63377982F976 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:03,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: E2161B6282074C52987A63377982F976
2016-05-18 00:58:03,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:03,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:03,913 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:03,913 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533083687_0001_1_00_000000_0
2016-05-18 00:58:03,915 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:03,915 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - E2161B6282074C52987A63377982F976 initialized RecordReader from event
2016-05-18 00:58:03,917 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:03,917 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: E2161B6282074C52987A63377982F976
2016-05-18 00:58:03,919 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_-1"], id: 81DF67B98E03441EB708F5D8A7622FBF
2016-05-18 00:58:03,920 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_-1"]
2016-05-18 00:58:03,921 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:03,921 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A02AE19A87E4469A8A051FBCF8C396F9, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:03,925 INFO output.MROutput (MROutput.java:close(526)) - 81DF67B98E03441EB708F5D8A7622FBF closed
2016-05-18 00:58:03,925 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533083687_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:03,926 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533083687_0001_1_00_000000_0
2016-05-18 00:58:03,926 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A02AE19A87E4469A8A051FBCF8C396F9, mem on close (mb), free: 285, total: 359, max: 672
2016-05-18 00:58:03,927 INFO output.MROutput (MROutput.java:close(526)) - 81DF67B98E03441EB708F5D8A7622FBF closed
2016-05-18 00:58:03,927 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:03 Completed running task attempt: attempt_1463533083687_0001_1_00_000000_0
2016-05-18 00:58:03,928 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533083687_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:03,928 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533083687_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:03,929 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533083687_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533083921, Process_Duration=5, Process_End_Time=1463533083926, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:03,930 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533083687_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:03,930 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:03,930 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:03,930 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:03,930 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533083687_0001_00_000001
2016-05-18 00:58:03,931 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083687_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A02AE19A87E4469A8A051FBCF8C396F9, taskAttemptId=attempt_1463533083687_0001_1_00_000000_0, creationTime=1463533083879, allocationTime=1463533083882, startTime=1463533083885, finishTime=1463533083930, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:03,931 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533083687_0001_1_00_000000_0
2016-05-18 00:58:03,932 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083687_0001_1][Event:TASK_FINISHED]: vertexName=A02AE19A87E4469A8A051FBCF8C396F9, taskId=task_1463533083687_0001_1_00_000000, startTime=1463533083885, finishTime=1463533083932, timeTaken=47, status=SUCCEEDED, successfulAttemptID=attempt_1463533083687_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533083921, Process_Duration=5, Process_End_Time=1463533083926, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:03,933 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:03,933 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9]
2016-05-18 00:58:03,935 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083687_0001_1][Event:VERTEX_FINISHED]: vertexName=A02AE19A87E4469A8A051FBCF8C396F9, vertexId=vertex_1463533083687_0001_1_00, initRequestedTime=1463533083859, initedTime=1463533083870, startRequestedTime=1463533083861, startedTime=1463533083875, finishTime=1463533083933, timeTaken=58, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533083921, Process_Duration=5, Process_End_Time=1463533083926, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533083885, firstTasksToStart=[ task_1463533083687_0001_1_00_000000 ], lastTaskFinishTime=1463533083930, lastTasksToFinish=[ task_1463533083687_0001_1_00_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533083687_0001_1_00_000000 ], longestDurationTasks=[ task_1463533083687_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:03,935 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:03,935 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083687_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533083687_0001_00_000001, stoppedTime=1463533083935, exitStatus=0
2016-05-18 00:58:03,936 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:03,936 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:03,936 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533083687_0001_1
2016-05-18 00:58:03,936 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083687_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533083687_0001_1
2016-05-18 00:58:03,937 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 81DF67B98E03441EB708F5D8A7622FBF for vertex: vertex_1463533083687_0001_1_00 [A02AE19A87E4469A8A051FBCF8C396F9], outputName: 81DF67B98E03441EB708F5D8A7622FBF
2016-05-18 00:58:03,941 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533083687_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:03,941 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533083687_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:03,941 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533083687_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:03,942 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533083687_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:03,942 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533083687_0001_00_000001, Exiting
2016-05-18 00:58:03,942 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533083687_0001_00_000001
2016-05-18 00:58:03,942 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533083687_0001_00_000001
2016-05-18 00:58:03,944 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533083687_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:03,944 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533083687_0001_00_000001 completed successfully
2016-05-18 00:58:03,945 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:81DF67B98E03441EB708F5D8A7622FBF of vertex/vertexGroup:A02AE19A87E4469A8A051FBCF8C396F9 isVertexGroupOutput:false
2016-05-18 00:58:03,945 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:03,946 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533083687_0001_1, queueSize=0
2016-05-18 00:58:03,947 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533083687_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533083687_0001_1, startTime=1463533083858, finishTime=1463533083945, timeTaken=87, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=150, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533083921, Process_Duration=5, Process_End_Time=1463533083926, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:03,947 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533083687_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:03,947 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533083687_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:03 Completed Dag: dag_1463533083687_0001_1
2016-05-18 00:58:03,948 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533083687_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:03,948 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533083687_0001_1_post
2016-05-18 00:58:03,948 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:03,948 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:03,948 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:03,948 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[6D1B6F8AA2EA404697C22374F0029574/F95E85D0E47C41199E90ECD2B0389EB4] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1, with id=dag_1463533083687_0001_1
2016-05-18 00:58:03,948 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[6D1B6F8AA2EA404697C22374F0029574/F95E85D0E47C41199E90ECD2B0389EB4] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1, with id=dag_1463533083687_0001_1
2016-05-18 00:58:03,958 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1, applicationId=application_1463533083687_0001
2016-05-18 00:58:03,960 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:03,961 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:03,961 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:03,961 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:03,961 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:03,961 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:03,962 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:03,962 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:03,962 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:03,963 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:03,963 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:03,963 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:03,964 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:03,965 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:03,968 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/6D1B6F8AA2EA404697C22374F0029574/F95E85D0E47C41199E90ECD2B0389EB4/.tez/application_1463533083687_0001
2016-05-18 00:58:03,968 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_-1/_temporary
2016-05-18 00:58:03,969 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.308
2016-05-18 00:58:04,045 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_UNKNOWN_-1
2016-05-18 00:58:04,071 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:04,075 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:04,114 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.007
2016-05-18 00:58:04,120 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.013
2016-05-18 00:58:04,120 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 00:58:04,121 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 00:58:04,121 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_UNKNOWN_'field2'
2016-05-18 00:58:04,147 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:04,147 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:04,191 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-18 00:58:04,203 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-05-18 00:58:04,203 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:04,204 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:04,204 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:04,221 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:04,222 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_'field'
2016-05-18 00:58:04,233 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:04,234 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:04,234 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_'field'"]
2016-05-18 00:58:04,234 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:04,234 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:04,234 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:04,235 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ESULTS.-1_UNKNOWN_'field'
2016-05-18 00:58:04,235 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:04,249 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:04,255 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:04,255 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:04,255 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:04,265 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/81226FBE69EA42D2AB332B2A0220883E/C3ECC7CBD6B545CCB57D3CCAD3A71A43/.tez/application_1463533084255_0001 doesn't exist and is created
2016-05-18 00:58:04,279 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:04,279 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:04,281 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/81226FBE69EA42D2AB332B2A0220883E/C3ECC7CBD6B545CCB57D3CCAD3A71A43/.tez/application_1463533084255_0001_wd
2016-05-18 00:58:04,292 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533084255_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:04,293 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:04,293 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:04,294 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:04,295 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:04,295 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:04,295 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:04,296 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:04,298 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:04,298 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:04,299 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:04,299 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:04,299 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:04,299 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:04,300 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:04,300 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:04,300 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:04,302 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:04,302 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:04,302 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:04,304 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:04,305 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:04,305 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:04,305 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/81226FBE69EA42D2AB332B2A0220883E/C3ECC7CBD6B545CCB57D3CCAD3A71A43/.tez/application_1463533084255_0001_wd/localmode-log-dir
2016-05-18 00:58:04,306 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/81226FBE69EA42D2AB332B2A0220883E/C3ECC7CBD6B545CCB57D3CCAD3A71A43/.tez/application_1463533084255_0001_wd/localmode-log-dir/history.txt.appattempt_1463533084255_0001_000000, maxErrors=10
2016-05-18 00:58:04,306 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/81226FBE69EA42D2AB332B2A0220883E/C3ECC7CBD6B545CCB57D3CCAD3A71A43/.tez/application_1463533084255_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:04,306 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533084255_0001_000000, appSubmitTime=1463533084292, launchTime=1463533084292
2016-05-18 00:58:04,310 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:04,315 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:45129
2016-05-18 00:58:04,316 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533084255
2016-05-18 00:58:04,317 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533084255_0001_000000, startTime=1463533084317
2016-05-18 00:58:04,317 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:04,380 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:04,380 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:04,380 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_'field', applicationId=application_1463533084255_0001, dagName=[81226FBE69EA42D2AB332B2A0220883E/C3ECC7CBD6B545CCB57D3CCAD3A71A43] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field'
2016-05-18 00:58:04,398 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [81226FBE69EA42D2AB332B2A0220883E/C3ECC7CBD6B545CCB57D3CCAD3A71A43] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field'
2016-05-18 00:58:04,398 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:04,399 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533084255_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/81226FBE69EA42D2AB332B2A0220883E/C3ECC7CBD6B545CCB57D3CCAD3A71A43/.tez/application_1463533084255_0001_wd/localmode-log-dir/dag_1463533084255_0001_1.dot
2016-05-18 00:58:04,400 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/81226FBE69EA42D2AB332B2A0220883E/C3ECC7CBD6B545CCB57D3CCAD3A71A43/.tez/application_1463533084255_0001_wd/localmode-log-dir/dag_1463533084255_0001_1-tez-dag.pb.txt
2016-05-18 00:58:04,406 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533084255_0001_1
2016-05-18 00:58:04,406 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:04,406 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [81226FBE69EA42D2AB332B2A0220883E/C3ECC7CBD6B545CCB57D3CCAD3A71A43] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field'
2016-05-18 00:58:04 Running Dag: dag_1463533084255_0001_1
2016-05-18 00:58:04,416 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084255_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533084255_0001_1, submitTime=1463533084398
2016-05-18 00:58:04,417 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/81226FBE69EA42D2AB332B2A0220883E/C3ECC7CBD6B545CCB57D3CCAD3A71A43/.tez/application_1463533084255_0001/tez-conf.pb" } size: 36913 timestamp: 1463533084000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:04,417 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34]
2016-05-18 00:58:04,418 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34]
2016-05-18 00:58:04,418 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:04,418 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:04,418 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084255_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533084255_0001_1, initTime=1463533084418
2016-05-18 00:58:04,418 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533084255_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:04,419 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_'field', applicationId=application_1463533084255_0001, dagName=[81226FBE69EA42D2AB332B2A0220883E/C3ECC7CBD6B545CCB57D3CCAD3A71A43] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field'
2016-05-18 00:58:04,420 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:04,422 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084255_0001_1][Event:DAG_STARTED]: dagID=dag_1463533084255_0001_1, startTime=1463533084422
2016-05-18 00:58:04,422 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533084255_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:04,422 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: BF27CA1BB3E34946B00472A0F5BA2A34 : {25C45CB6394F468090F1900CC2D30C9E={InputName=25C45CB6394F468090F1900CC2D30C9E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:04,422 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 25C45CB6394F468090F1900CC2D30C9E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:04,422 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34]
2016-05-18 00:58:04,422 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34]
2016-05-18 00:58:04,423 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34]
2016-05-18 00:58:04,423 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34]
2016-05-18 00:58:04,423 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:04,424 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 25C45CB6394F468090F1900CC2D30C9E on vertex vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34]
2016-05-18 00:58:04,426 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 25C45CB6394F468090F1900CC2D30C9E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:04,428 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:04,428 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:04,428 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:04,429 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:04,429 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 25C45CB6394F468090F1900CC2D30C9E on vertex vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34]
2016-05-18 00:58:04,430 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:04,430 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {25C45CB6394F468090F1900CC2D30C9E=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:04,431 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34] parallelism set to 1
2016-05-18 00:58:04,431 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:04,432 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34], numAdditionalOutputs=1
2016-05-18 00:58:04,432 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=96F79494A0B94CD2B6E0F70907CCCE2E, vertex=vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:04,433 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for BF27CA1BB3E34946B00472A0F5BA2A34:96F79494A0B94CD2B6E0F70907CCCE2E using oldmapred API
2016-05-18 00:58:04,434 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=96F79494A0B94CD2B6E0F70907CCCE2E, vertexName=BF27CA1BB3E34946B00472A0F5BA2A34, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:04,434 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084255_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BF27CA1BB3E34946B00472A0F5BA2A34, vertexId=vertex_1463533084255_0001_1_00, initRequestedTime=1463533084422, initedTime=1463533084432, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:04,434 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34] with distanceFromRoot: 0
2016-05-18 00:58:04,435 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:04,435 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084255_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533084255_0001_1_00, startRequestedTime=1463533084423, startedTime=1463533084435
2016-05-18 00:58:04,436 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084255_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533084255_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:04,436 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:04,439 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in BF27CA1BB3E34946B00472A0F5BA2A34
2016-05-18 00:58:04,439 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34]
2016-05-18 00:58:04,440 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084255_0001_1][Event:TASK_STARTED]: vertexName=BF27CA1BB3E34946B00472A0F5BA2A34, taskId=task_1463533084255_0001_1_00_000000, scheduledTime=1463533084440, launchTime=1463533084440
2016-05-18 00:58:04,441 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34]
2016-05-18 00:58:04,442 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:04,445 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533084255_0001_00_000001, launchTime=1463533084444
2016-05-18 00:58:04,445 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533084255_0001_00_000001
2016-05-18 00:58:04,446 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533084255_0001_1_00_000000_0] started. Is using containerId: [container_1463533084255_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:04,446 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084255_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BF27CA1BB3E34946B00472A0F5BA2A34, taskAttemptId=attempt_1463533084255_0001_1_00_000000_0, startTime=1463533084446, containerId=container_1463533084255_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:04,447 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533084255_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533084255_0001_1_00_000000_0
2016-05-18 00:58:04,447 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533084255_0001, with dag id: dag_1463533084255_0001_1
2016-05-18 00:58:04 Starting to run new task attempt: attempt_1463533084255_0001_1_00_000000_0
2016-05-18 00:58:04,447 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533084255_0001_1_00_000000_0
2016-05-18 00:58:04,448 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:04,449 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:04,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [81226FBE69EA42D2AB332B2A0220883E/C3ECC7CBD6B545CCB57D3CCAD3A71A43] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field', VertexName: BF27CA1BB3E34946B00472A0F5BA2A34, VertexParallelism: 1, TaskAttemptID:attempt_1463533084255_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=25C45CB6394F468090F1900CC2D30C9E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=96F79494A0B94CD2B6E0F70907CCCE2E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:04,449 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:04,451 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533084255_0001_1_00_000000_0
2016-05-18 00:58:04,454 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533084255_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:04,454 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533084255_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:04,459 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:04,460 INFO input.MRInput (MRInput.java:initialize(440)) - 25C45CB6394F468090F1900CC2D30C9E using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:04,460 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 25C45CB6394F468090F1900CC2D30C9E
2016-05-18 00:58:04,468 INFO output.MROutput (MROutput.java:initialize(421)) - 96F79494A0B94CD2B6E0F70907CCCE2E: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:04,476 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BF27CA1BB3E34946B00472A0F5BA2A34, ordinal: 0
2016-05-18 00:58:04,476 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BF27CA1BB3E34946B00472A0F5BA2A34, mem on start (mb), free: 175, total: 359, max: 672
2016-05-18 00:58:04,477 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:04,478 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:04,478 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:04,478 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:04,478 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[25C45CB6394F468090F1900CC2D30C9E:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [96F79494A0B94CD2B6E0F70907CCCE2E:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:04,480 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:04,480 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:04,480 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:04,480 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[25C45CB6394F468090F1900CC2D30C9E:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [96F79494A0B94CD2B6E0F70907CCCE2E:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:04,482 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:04,482 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 25C45CB6394F468090F1900CC2D30C9E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:04,482 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 25C45CB6394F468090F1900CC2D30C9E
2016-05-18 00:58:04,482 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:04,482 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:04,483 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:04,483 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533084255_0001_1_00_000000_0
2016-05-18 00:58:04,485 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:04,485 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 25C45CB6394F468090F1900CC2D30C9E initialized RecordReader from event
2016-05-18 00:58:04,487 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:04,487 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 25C45CB6394F468090F1900CC2D30C9E
2016-05-18 00:58:04,488 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_'field'"], id: 96F79494A0B94CD2B6E0F70907CCCE2E
2016-05-18 00:58:04,488 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_'field'"]
2016-05-18 00:58:04,488 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:04,489 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BF27CA1BB3E34946B00472A0F5BA2A34, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:04,492 INFO output.MROutput (MROutput.java:close(526)) - 96F79494A0B94CD2B6E0F70907CCCE2E closed
2016-05-18 00:58:04,493 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533084255_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:04,493 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533084255_0001_1_00_000000_0
2016-05-18 00:58:04,494 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BF27CA1BB3E34946B00472A0F5BA2A34, mem on close (mb), free: 173, total: 359, max: 672
2016-05-18 00:58:04,494 INFO output.MROutput (MROutput.java:close(526)) - 96F79494A0B94CD2B6E0F70907CCCE2E closed
2016-05-18 00:58:04,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:04 Completed running task attempt: attempt_1463533084255_0001_1_00_000000_0
2016-05-18 00:58:04,495 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533084255_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:04,495 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533084255_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:04,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533084255_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533084489, Process_Duration=4, Process_End_Time=1463533084493, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:04,497 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533084255_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:04,497 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:04,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084255_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BF27CA1BB3E34946B00472A0F5BA2A34, taskAttemptId=attempt_1463533084255_0001_1_00_000000_0, creationTime=1463533084440, allocationTime=1463533084444, startTime=1463533084446, finishTime=1463533084497, timeTaken=51, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:04,497 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:04,498 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533084255_0001_1_00_000000_0
2016-05-18 00:58:04,499 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:04,499 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533084255_0001_00_000001
2016-05-18 00:58:04,499 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084255_0001_1][Event:TASK_FINISHED]: vertexName=BF27CA1BB3E34946B00472A0F5BA2A34, taskId=task_1463533084255_0001_1_00_000000, startTime=1463533084446, finishTime=1463533084499, timeTaken=53, status=SUCCEEDED, successfulAttemptID=attempt_1463533084255_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533084489, Process_Duration=4, Process_End_Time=1463533084493, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:04,500 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:04,500 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34]
2016-05-18 00:58:04,501 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084255_0001_1][Event:VERTEX_FINISHED]: vertexName=BF27CA1BB3E34946B00472A0F5BA2A34, vertexId=vertex_1463533084255_0001_1_00, initRequestedTime=1463533084422, initedTime=1463533084432, startRequestedTime=1463533084423, startedTime=1463533084435, finishTime=1463533084500, timeTaken=65, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533084489, Process_Duration=4, Process_End_Time=1463533084493, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533084446, firstTasksToStart=[ task_1463533084255_0001_1_00_000000 ], lastTaskFinishTime=1463533084497, lastTasksToFinish=[ task_1463533084255_0001_1_00_000000 ], minTaskDuration=51, maxTaskDuration=51, avgTaskDuration=51.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533084255_0001_1_00_000000 ], longestDurationTasks=[ task_1463533084255_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:04,502 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:04,502 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084255_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533084255_0001_00_000001, stoppedTime=1463533084502, exitStatus=0
2016-05-18 00:58:04,502 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:04,502 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:04,502 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533084255_0001_1
2016-05-18 00:58:04,502 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084255_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533084255_0001_1
2016-05-18 00:58:04,503 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 96F79494A0B94CD2B6E0F70907CCCE2E for vertex: vertex_1463533084255_0001_1_00 [BF27CA1BB3E34946B00472A0F5BA2A34], outputName: 96F79494A0B94CD2B6E0F70907CCCE2E
2016-05-18 00:58:04,505 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533084255_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:04,505 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533084255_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:04,510 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533084255_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:04,510 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533084255_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:04,510 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533084255_0001_00_000001, Exiting
2016-05-18 00:58:04,510 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533084255_0001_00_000001
2016-05-18 00:58:04,510 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533084255_0001_00_000001
2016-05-18 00:58:04,510 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:96F79494A0B94CD2B6E0F70907CCCE2E of vertex/vertexGroup:BF27CA1BB3E34946B00472A0F5BA2A34 isVertexGroupOutput:false
2016-05-18 00:58:04,510 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:04,511 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533084255_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:04,511 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533084255_0001_00_000001 completed successfully
2016-05-18 00:58:04,512 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533084255_0001_1, queueSize=0
2016-05-18 00:58:04,512 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084255_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533084255_0001_1, startTime=1463533084420, finishTime=1463533084511, timeTaken=91, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=160, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533084489, Process_Duration=4, Process_End_Time=1463533084493, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:04,513 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533084255_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:04,513 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533084255_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:04 Completed Dag: dag_1463533084255_0001_1
2016-05-18 00:58:04,513 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533084255_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:04,513 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533084255_0001_1_post
2016-05-18 00:58:04,513 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:04,513 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:04,514 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:04,514 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[81226FBE69EA42D2AB332B2A0220883E/C3ECC7CBD6B545CCB57D3CCAD3A71A43] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field', with id=dag_1463533084255_0001_1
2016-05-18 00:58:04,514 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[81226FBE69EA42D2AB332B2A0220883E/C3ECC7CBD6B545CCB57D3CCAD3A71A43] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field', with id=dag_1463533084255_0001_1
2016-05-18 00:58:04,521 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_'field', applicationId=application_1463533084255_0001
2016-05-18 00:58:04,523 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:04,524 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:04,524 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:04,524 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:04,524 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:04,524 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:04,525 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:04,525 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:04,525 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:04,525 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:04,526 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:04,526 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:04,526 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:04,526 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:04,530 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/81226FBE69EA42D2AB332B2A0220883E/C3ECC7CBD6B545CCB57D3CCAD3A71A43/.tez/application_1463533084255_0001
2016-05-18 00:58:04,530 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_'field'/_temporary
2016-05-18 00:58:04,531 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.297
2016-05-18 00:58:04,608 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_UNKNOWN_'field'
2016-05-18 00:58:04,639 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:04,639 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:04,686 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-18 00:58:04,690 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-05-18 00:58:04,690 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:04,690 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:04,690 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:04,706 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:04,707 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_RESULTS
2016-05-18 00:58:04,719 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:04,720 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:04,720 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_RESULTS"]
2016-05-18 00:58:04,720 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:04,721 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:04,721 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:04,722 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ESULTS.-1_UNKNOWN_RESULTS
2016-05-18 00:58:04,723 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:04,764 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:04,770 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:04,770 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:04,770 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:04,781 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/190B3EC897FE46CDA8E0735506434797/EE803AEF54A14706A1BE16C34DE39170/.tez/application_1463533084770_0001 doesn't exist and is created
2016-05-18 00:58:04,797 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:04,797 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:04,799 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/190B3EC897FE46CDA8E0735506434797/EE803AEF54A14706A1BE16C34DE39170/.tez/application_1463533084770_0001_wd
2016-05-18 00:58:04,810 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533084770_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:04,810 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:04,810 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:04,812 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:04,812 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:04,812 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:04,812 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:04,813 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:04,815 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:04,815 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:04,815 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:04,816 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:04,816 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:04,816 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:04,817 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:04,817 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:04,817 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:04,817 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:04,818 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:04,818 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:04,819 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:04,820 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:04,820 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:04,821 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/190B3EC897FE46CDA8E0735506434797/EE803AEF54A14706A1BE16C34DE39170/.tez/application_1463533084770_0001_wd/localmode-log-dir
2016-05-18 00:58:04,821 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/190B3EC897FE46CDA8E0735506434797/EE803AEF54A14706A1BE16C34DE39170/.tez/application_1463533084770_0001_wd/localmode-log-dir/history.txt.appattempt_1463533084770_0001_000000, maxErrors=10
2016-05-18 00:58:04,822 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/190B3EC897FE46CDA8E0735506434797/EE803AEF54A14706A1BE16C34DE39170/.tez/application_1463533084770_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:04,822 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533084770_0001_000000, appSubmitTime=1463533084809, launchTime=1463533084809
2016-05-18 00:58:04,826 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:04,832 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49969
2016-05-18 00:58:04,833 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533084770
2016-05-18 00:58:04,834 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533084770_0001_000000, startTime=1463533084834
2016-05-18 00:58:04,835 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:04,898 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:04,898 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:04,898 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_RESULTS, applicationId=application_1463533084770_0001, dagName=[190B3EC897FE46CDA8E0735506434797/EE803AEF54A14706A1BE16C34DE39170] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS
2016-05-18 00:58:04,920 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [190B3EC897FE46CDA8E0735506434797/EE803AEF54A14706A1BE16C34DE39170] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS
2016-05-18 00:58:04,921 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:04,921 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533084770_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/190B3EC897FE46CDA8E0735506434797/EE803AEF54A14706A1BE16C34DE39170/.tez/application_1463533084770_0001_wd/localmode-log-dir/dag_1463533084770_0001_1.dot
2016-05-18 00:58:04,922 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/190B3EC897FE46CDA8E0735506434797/EE803AEF54A14706A1BE16C34DE39170/.tez/application_1463533084770_0001_wd/localmode-log-dir/dag_1463533084770_0001_1-tez-dag.pb.txt
2016-05-18 00:58:04,932 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533084770_0001_1
2016-05-18 00:58:04,932 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:04,932 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [190B3EC897FE46CDA8E0735506434797/EE803AEF54A14706A1BE16C34DE39170] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS
2016-05-18 00:58:04 Running Dag: dag_1463533084770_0001_1
2016-05-18 00:58:04,941 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084770_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533084770_0001_1, submitTime=1463533084920
2016-05-18 00:58:04,942 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/190B3EC897FE46CDA8E0735506434797/EE803AEF54A14706A1BE16C34DE39170/.tez/application_1463533084770_0001/tez-conf.pb" } size: 36913 timestamp: 1463533084000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:04,943 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF]
2016-05-18 00:58:04,943 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF]
2016-05-18 00:58:04,943 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:04,944 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:04,944 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084770_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533084770_0001_1, initTime=1463533084944
2016-05-18 00:58:04,945 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533084770_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:04,945 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:04,946 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084770_0001_1][Event:DAG_STARTED]: dagID=dag_1463533084770_0001_1, startTime=1463533084946
2016-05-18 00:58:04,946 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533084770_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:04,946 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: C826131F9F7E43668FDA3E1EAFFB89EF : {7E213D863F4B49FBA9DB0284E8165C30={InputName=7E213D863F4B49FBA9DB0284E8165C30}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:04,946 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 7E213D863F4B49FBA9DB0284E8165C30, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:04,946 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF]
2016-05-18 00:58:04,946 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF]
2016-05-18 00:58:04,947 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF]
2016-05-18 00:58:04,947 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF]
2016-05-18 00:58:04,947 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:04,947 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 7E213D863F4B49FBA9DB0284E8165C30 on vertex vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF]
2016-05-18 00:58:04,948 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_RESULTS, applicationId=application_1463533084770_0001, dagName=[190B3EC897FE46CDA8E0735506434797/EE803AEF54A14706A1BE16C34DE39170] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS
2016-05-18 00:58:04,949 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 7E213D863F4B49FBA9DB0284E8165C30 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:04,951 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:04,951 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:04,951 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:04,951 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:04,952 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 7E213D863F4B49FBA9DB0284E8165C30 on vertex vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF]
2016-05-18 00:58:04,953 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:04,953 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {7E213D863F4B49FBA9DB0284E8165C30=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:04,955 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF] parallelism set to 1
2016-05-18 00:58:04,955 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:04,955 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF], numAdditionalOutputs=1
2016-05-18 00:58:04,955 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=3FE0DDEF825A4695954839916BB3C6CB, vertex=vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:04,957 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for C826131F9F7E43668FDA3E1EAFFB89EF:3FE0DDEF825A4695954839916BB3C6CB using oldmapred API
2016-05-18 00:58:04,957 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=3FE0DDEF825A4695954839916BB3C6CB, vertexName=C826131F9F7E43668FDA3E1EAFFB89EF, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:04,958 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084770_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C826131F9F7E43668FDA3E1EAFFB89EF, vertexId=vertex_1463533084770_0001_1_00, initRequestedTime=1463533084946, initedTime=1463533084955, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:04,958 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF] with distanceFromRoot: 0
2016-05-18 00:58:04,958 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:04,959 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084770_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533084770_0001_1_00, startRequestedTime=1463533084947, startedTime=1463533084959
2016-05-18 00:58:04,959 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084770_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533084770_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:04,959 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:04,960 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in C826131F9F7E43668FDA3E1EAFFB89EF
2016-05-18 00:58:04,961 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF]
2016-05-18 00:58:04,961 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084770_0001_1][Event:TASK_STARTED]: vertexName=C826131F9F7E43668FDA3E1EAFFB89EF, taskId=task_1463533084770_0001_1_00_000000, scheduledTime=1463533084961, launchTime=1463533084961
2016-05-18 00:58:04,962 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF]
2016-05-18 00:58:04,965 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:04,968 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533084770_0001_00_000001, launchTime=1463533084967
2016-05-18 00:58:04,968 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533084770_0001_00_000001
2016-05-18 00:58:04,971 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533084770_0001_1_00_000000_0] started. Is using containerId: [container_1463533084770_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:04,971 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084770_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C826131F9F7E43668FDA3E1EAFFB89EF, taskAttemptId=attempt_1463533084770_0001_1_00_000000_0, startTime=1463533084970, containerId=container_1463533084770_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:04,973 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533084770_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533084770_0001_1_00_000000_0
2016-05-18 00:58:04,973 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533084770_0001, with dag id: dag_1463533084770_0001_1
2016-05-18 00:58:04 Starting to run new task attempt: attempt_1463533084770_0001_1_00_000000_0
2016-05-18 00:58:04,974 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533084770_0001_1_00_000000_0
2016-05-18 00:58:04,974 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:04,975 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:04,976 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [190B3EC897FE46CDA8E0735506434797/EE803AEF54A14706A1BE16C34DE39170] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS, VertexName: C826131F9F7E43668FDA3E1EAFFB89EF, VertexParallelism: 1, TaskAttemptID:attempt_1463533084770_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7E213D863F4B49FBA9DB0284E8165C30, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=3FE0DDEF825A4695954839916BB3C6CB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:04,976 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:04,977 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533084770_0001_1_00_000000_0
2016-05-18 00:58:04,978 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533084770_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:04,979 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533084770_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:04,984 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:04,984 INFO input.MRInput (MRInput.java:initialize(440)) - 7E213D863F4B49FBA9DB0284E8165C30 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:04,984 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 7E213D863F4B49FBA9DB0284E8165C30
2016-05-18 00:58:04,995 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C826131F9F7E43668FDA3E1EAFFB89EF, ordinal: 0
2016-05-18 00:58:04,995 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C826131F9F7E43668FDA3E1EAFFB89EF, mem on start (mb), free: 294, total: 378, max: 672
2016-05-18 00:58:04,995 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:04,996 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:04,996 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:04,999 INFO output.MROutput (MROutput.java:initialize(421)) - 3FE0DDEF825A4695954839916BB3C6CB: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:05,000 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:05,000 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[7E213D863F4B49FBA9DB0284E8165C30:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [3FE0DDEF825A4695954839916BB3C6CB:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:05,000 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:05,000 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:05,000 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:05,001 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[7E213D863F4B49FBA9DB0284E8165C30:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [3FE0DDEF825A4695954839916BB3C6CB:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:05,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:05,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 7E213D863F4B49FBA9DB0284E8165C30 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:05,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 7E213D863F4B49FBA9DB0284E8165C30
2016-05-18 00:58:05,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:05,003 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:05,003 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:05,003 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533084770_0001_1_00_000000_0
2016-05-18 00:58:05,005 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:05,005 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 7E213D863F4B49FBA9DB0284E8165C30 initialized RecordReader from event
2016-05-18 00:58:05,006 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:05,007 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 7E213D863F4B49FBA9DB0284E8165C30
2016-05-18 00:58:05,007 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_RESULTS"], id: 3FE0DDEF825A4695954839916BB3C6CB
2016-05-18 00:58:05,007 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_RESULTS"]
2016-05-18 00:58:05,007 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:05,008 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C826131F9F7E43668FDA3E1EAFFB89EF, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:05,011 INFO output.MROutput (MROutput.java:close(526)) - 3FE0DDEF825A4695954839916BB3C6CB closed
2016-05-18 00:58:05,011 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533084770_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:05,012 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533084770_0001_1_00_000000_0
2016-05-18 00:58:05,012 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C826131F9F7E43668FDA3E1EAFFB89EF, mem on close (mb), free: 292, total: 378, max: 672
2016-05-18 00:58:05,012 INFO output.MROutput (MROutput.java:close(526)) - 3FE0DDEF825A4695954839916BB3C6CB closed
2016-05-18 00:58:05,013 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:05 Completed running task attempt: attempt_1463533084770_0001_1_00_000000_0
2016-05-18 00:58:05,014 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533084770_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:05,015 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533084770_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:05,015 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533084770_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533085008, Process_Duration=4, Process_End_Time=1463533085012, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:05,016 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533084770_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:05,016 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:05,016 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084770_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C826131F9F7E43668FDA3E1EAFFB89EF, taskAttemptId=attempt_1463533084770_0001_1_00_000000_0, creationTime=1463533084962, allocationTime=1463533084966, startTime=1463533084970, finishTime=1463533085016, timeTaken=46, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:05,016 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:05,017 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:05,017 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533084770_0001_1_00_000000_0
2016-05-18 00:58:05,017 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533084770_0001_00_000001
2016-05-18 00:58:05,018 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084770_0001_1][Event:TASK_FINISHED]: vertexName=C826131F9F7E43668FDA3E1EAFFB89EF, taskId=task_1463533084770_0001_1_00_000000, startTime=1463533084970, finishTime=1463533085017, timeTaken=47, status=SUCCEEDED, successfulAttemptID=attempt_1463533084770_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533085008, Process_Duration=4, Process_End_Time=1463533085012, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:05,018 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:05,018 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF]
2016-05-18 00:58:05,020 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084770_0001_1][Event:VERTEX_FINISHED]: vertexName=C826131F9F7E43668FDA3E1EAFFB89EF, vertexId=vertex_1463533084770_0001_1_00, initRequestedTime=1463533084946, initedTime=1463533084955, startRequestedTime=1463533084947, startedTime=1463533084959, finishTime=1463533085018, timeTaken=59, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533085008, Process_Duration=4, Process_End_Time=1463533085012, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533084970, firstTasksToStart=[ task_1463533084770_0001_1_00_000000 ], lastTaskFinishTime=1463533085016, lastTasksToFinish=[ task_1463533084770_0001_1_00_000000 ], minTaskDuration=46, maxTaskDuration=46, avgTaskDuration=46.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533084770_0001_1_00_000000 ], longestDurationTasks=[ task_1463533084770_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:05,020 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:05,020 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084770_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533084770_0001_00_000001, stoppedTime=1463533085020, exitStatus=0
2016-05-18 00:58:05,021 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:05,021 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:05,021 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533084770_0001_1
2016-05-18 00:58:05,021 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084770_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533084770_0001_1
2016-05-18 00:58:05,021 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533084770_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:05,021 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 3FE0DDEF825A4695954839916BB3C6CB for vertex: vertex_1463533084770_0001_1_00 [C826131F9F7E43668FDA3E1EAFFB89EF], outputName: 3FE0DDEF825A4695954839916BB3C6CB
2016-05-18 00:58:05,021 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533084770_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:05,028 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:3FE0DDEF825A4695954839916BB3C6CB of vertex/vertexGroup:C826131F9F7E43668FDA3E1EAFFB89EF isVertexGroupOutput:false
2016-05-18 00:58:05,028 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533084770_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:05,028 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:05,029 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533084770_0001_1, queueSize=0
2016-05-18 00:58:05,030 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533084770_0001_00_000001: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:05,030 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533084770_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533084770_0001_1, startTime=1463533084945, finishTime=1463533085028, timeTaken=83, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=190, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533085008, Process_Duration=4, Process_End_Time=1463533085012, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:05,030 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533084770_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:05,030 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533084770_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:05,030 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533084770_0001_00_000001, Exiting
2016-05-18 00:58:05,031 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533084770_0001_00_000001
2016-05-18 00:58:05 Completed Dag: dag_1463533084770_0001_1
2016-05-18 00:58:05,031 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533084770_0001_00_000001
2016-05-18 00:58:05,031 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533084770_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:05,031 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533084770_0001_1_post
2016-05-18 00:58:05,031 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:05,031 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:05,031 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:05,031 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533084770_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:05,032 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[190B3EC897FE46CDA8E0735506434797/EE803AEF54A14706A1BE16C34DE39170] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS, with id=dag_1463533084770_0001_1
2016-05-18 00:58:05,032 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[190B3EC897FE46CDA8E0735506434797/EE803AEF54A14706A1BE16C34DE39170] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS, with id=dag_1463533084770_0001_1
2016-05-18 00:58:05,032 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533084770_0001_00_000001 completed successfully
2016-05-18 00:58:05,043 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_RESULTS, applicationId=application_1463533084770_0001
2016-05-18 00:58:05,045 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:05,045 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:05,045 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:05,045 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:05,045 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:05,045 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:05,046 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:05,046 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:05,047 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:05,047 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:05,047 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:05,047 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:05,047 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:05,048 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:05,051 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/190B3EC897FE46CDA8E0735506434797/EE803AEF54A14706A1BE16C34DE39170/.tez/application_1463533084770_0001
2016-05-18 00:58:05,051 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_RESULTS/_temporary
2016-05-18 00:58:05,052 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.332
2016-05-18 00:58:05,133 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_UNKNOWN_RESULTS
2016-05-18 00:58:05,159 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:05,159 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:05,207 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-05-18 00:58:05,215 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.028
2016-05-18 00:58:05,216 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:05,216 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:05,216 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:05,232 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:05,232 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_ALL
2016-05-18 00:58:05,244 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:05,245 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:05,245 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_ALL"]
2016-05-18 00:58:05,245 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:05,246 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:05,246 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:05,247 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...d'_RESULTS.-1_UNKNOWN_ALL
2016-05-18 00:58:05,247 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:05,262 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:05,267 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:05,268 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:05,268 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:05,277 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/6F6264F24CAA4195958E2C669CFA6FA7/A0A71F48AFEE40FDB909B397FECEB029/.tez/application_1463533085267_0001 doesn't exist and is created
2016-05-18 00:58:05,294 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:05,294 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:05,296 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/6F6264F24CAA4195958E2C669CFA6FA7/A0A71F48AFEE40FDB909B397FECEB029/.tez/application_1463533085267_0001_wd
2016-05-18 00:58:05,307 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533085267_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:05,307 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:05,307 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:05,309 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:05,309 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:05,309 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:05,309 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:05,310 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:05,312 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:05,312 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:05,312 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:05,312 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:05,312 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:05,312 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:05,313 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:05,313 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:05,313 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:05,313 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:05,313 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:05,313 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:05,315 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:05,316 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:05,316 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:05,316 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/6F6264F24CAA4195958E2C669CFA6FA7/A0A71F48AFEE40FDB909B397FECEB029/.tez/application_1463533085267_0001_wd/localmode-log-dir
2016-05-18 00:58:05,316 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/6F6264F24CAA4195958E2C669CFA6FA7/A0A71F48AFEE40FDB909B397FECEB029/.tez/application_1463533085267_0001_wd/localmode-log-dir/history.txt.appattempt_1463533085267_0001_000000, maxErrors=10
2016-05-18 00:58:05,317 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/6F6264F24CAA4195958E2C669CFA6FA7/A0A71F48AFEE40FDB909B397FECEB029/.tez/application_1463533085267_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:05,317 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533085267_0001_000000, appSubmitTime=1463533085306, launchTime=1463533085306
2016-05-18 00:58:05,321 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:05,325 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:43775
2016-05-18 00:58:05,326 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533085267
2016-05-18 00:58:05,328 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533085267_0001_000000, startTime=1463533085328
2016-05-18 00:58:05,328 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:05,395 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:05,395 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:05,395 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL, applicationId=application_1463533085267_0001, dagName=[6F6264F24CAA4195958E2C669CFA6FA7/A0A71F48AFEE40FDB909B397FECEB029] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL
2016-05-18 00:58:05,412 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [6F6264F24CAA4195958E2C669CFA6FA7/A0A71F48AFEE40FDB909B397FECEB029] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL
2016-05-18 00:58:05,413 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:05,414 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533085267_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/6F6264F24CAA4195958E2C669CFA6FA7/A0A71F48AFEE40FDB909B397FECEB029/.tez/application_1463533085267_0001_wd/localmode-log-dir/dag_1463533085267_0001_1.dot
2016-05-18 00:58:05,414 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/6F6264F24CAA4195958E2C669CFA6FA7/A0A71F48AFEE40FDB909B397FECEB029/.tez/application_1463533085267_0001_wd/localmode-log-dir/dag_1463533085267_0001_1-tez-dag.pb.txt
2016-05-18 00:58:05,419 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533085267_0001_1
2016-05-18 00:58:05,419 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:05,419 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [6F6264F24CAA4195958E2C669CFA6FA7/A0A71F48AFEE40FDB909B397FECEB029] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL
2016-05-18 00:58:05 Running Dag: dag_1463533085267_0001_1
2016-05-18 00:58:05,430 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085267_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533085267_0001_1, submitTime=1463533085413
2016-05-18 00:58:05,431 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/6F6264F24CAA4195958E2C669CFA6FA7/A0A71F48AFEE40FDB909B397FECEB029/.tez/application_1463533085267_0001/tez-conf.pb" } size: 36913 timestamp: 1463533085000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:05,431 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B]
2016-05-18 00:58:05,431 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B]
2016-05-18 00:58:05,431 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:05,431 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:05,432 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085267_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533085267_0001_1, initTime=1463533085432
2016-05-18 00:58:05,432 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533085267_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:05,433 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:05,433 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085267_0001_1][Event:DAG_STARTED]: dagID=dag_1463533085267_0001_1, startTime=1463533085433
2016-05-18 00:58:05,433 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533085267_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:05,433 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL, applicationId=application_1463533085267_0001, dagName=[6F6264F24CAA4195958E2C669CFA6FA7/A0A71F48AFEE40FDB909B397FECEB029] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL
2016-05-18 00:58:05,433 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: E479C8CEB95643F48BE893CE33DC276B : {C90745CA98DB4D54A31B1B74478A93B8={InputName=C90745CA98DB4D54A31B1B74478A93B8}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:05,435 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: C90745CA98DB4D54A31B1B74478A93B8, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:05,435 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B]
2016-05-18 00:58:05,435 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B]
2016-05-18 00:58:05,435 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B]
2016-05-18 00:58:05,436 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B]
2016-05-18 00:58:05,437 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:05,437 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: C90745CA98DB4D54A31B1B74478A93B8 on vertex vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B]
2016-05-18 00:58:05,439 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input C90745CA98DB4D54A31B1B74478A93B8 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:05,440 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:05,440 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:05,441 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:05,441 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:05,441 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: C90745CA98DB4D54A31B1B74478A93B8 on vertex vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B]
2016-05-18 00:58:05,442 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:05,442 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {C90745CA98DB4D54A31B1B74478A93B8=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:05,443 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B] parallelism set to 1
2016-05-18 00:58:05,443 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:05,444 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B], numAdditionalOutputs=1
2016-05-18 00:58:05,444 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=F2D2E977F7584616B2D6EFA3D04FC993, vertex=vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:05,445 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for E479C8CEB95643F48BE893CE33DC276B:F2D2E977F7584616B2D6EFA3D04FC993 using oldmapred API
2016-05-18 00:58:05,446 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=F2D2E977F7584616B2D6EFA3D04FC993, vertexName=E479C8CEB95643F48BE893CE33DC276B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:05,447 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085267_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E479C8CEB95643F48BE893CE33DC276B, vertexId=vertex_1463533085267_0001_1_00, initRequestedTime=1463533085433, initedTime=1463533085444, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:05,447 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B] with distanceFromRoot: 0
2016-05-18 00:58:05,447 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:05,448 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085267_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533085267_0001_1_00, startRequestedTime=1463533085437, startedTime=1463533085447
2016-05-18 00:58:05,448 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085267_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533085267_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:05,448 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:05,448 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in E479C8CEB95643F48BE893CE33DC276B
2016-05-18 00:58:05,449 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B]
2016-05-18 00:58:05,449 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085267_0001_1][Event:TASK_STARTED]: vertexName=E479C8CEB95643F48BE893CE33DC276B, taskId=task_1463533085267_0001_1_00_000000, scheduledTime=1463533085449, launchTime=1463533085449
2016-05-18 00:58:05,450 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B]
2016-05-18 00:58:05,450 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:05,452 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533085267_0001_00_000001, launchTime=1463533085452
2016-05-18 00:58:05,453 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533085267_0001_00_000001
2016-05-18 00:58:05,454 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533085267_0001_1_00_000000_0] started. Is using containerId: [container_1463533085267_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:05,454 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085267_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E479C8CEB95643F48BE893CE33DC276B, taskAttemptId=attempt_1463533085267_0001_1_00_000000_0, startTime=1463533085453, containerId=container_1463533085267_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:05,455 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533085267_0001, with dag id: dag_1463533085267_0001_1
2016-05-18 00:58:05,455 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533085267_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533085267_0001_1_00_000000_0
2016-05-18 00:58:05 Starting to run new task attempt: attempt_1463533085267_0001_1_00_000000_0
2016-05-18 00:58:05,456 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533085267_0001_1_00_000000_0
2016-05-18 00:58:05,456 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:05,456 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:05,458 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6F6264F24CAA4195958E2C669CFA6FA7/A0A71F48AFEE40FDB909B397FECEB029] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL, VertexName: E479C8CEB95643F48BE893CE33DC276B, VertexParallelism: 1, TaskAttemptID:attempt_1463533085267_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C90745CA98DB4D54A31B1B74478A93B8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F2D2E977F7584616B2D6EFA3D04FC993, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:05,458 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:05,460 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533085267_0001_1_00_000000_0
2016-05-18 00:58:05,461 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533085267_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:05,462 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533085267_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:05,467 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:05,467 INFO input.MRInput (MRInput.java:initialize(440)) - C90745CA98DB4D54A31B1B74478A93B8 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:05,468 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: C90745CA98DB4D54A31B1B74478A93B8
2016-05-18 00:58:05,475 INFO output.MROutput (MROutput.java:initialize(421)) - F2D2E977F7584616B2D6EFA3D04FC993: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:05,476 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E479C8CEB95643F48BE893CE33DC276B, ordinal: 0
2016-05-18 00:58:05,477 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E479C8CEB95643F48BE893CE33DC276B, mem on start (mb), free: 205, total: 378, max: 672
2016-05-18 00:58:05,477 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:05,477 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:05,477 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:05,478 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:05,478 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[F2D2E977F7584616B2D6EFA3D04FC993:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [C90745CA98DB4D54A31B1B74478A93B8:INPUT:0:org.apache.tez.mapreduce.input.MRInput]
2016-05-18 00:58:05,478 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:05,478 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:05,479 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:05,479 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[F2D2E977F7584616B2D6EFA3D04FC993:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [C90745CA98DB4D54A31B1B74478A93B8:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0]
2016-05-18 00:58:05,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:05,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: C90745CA98DB4D54A31B1B74478A93B8
2016-05-18 00:58:05,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: C90745CA98DB4D54A31B1B74478A93B8 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:05,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:05,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:05,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:05,480 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533085267_0001_1_00_000000_0
2016-05-18 00:58:05,481 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:05,481 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - C90745CA98DB4D54A31B1B74478A93B8 initialized RecordReader from event
2016-05-18 00:58:05,483 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:05,484 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: C90745CA98DB4D54A31B1B74478A93B8
2016-05-18 00:58:05,484 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_ALL"], id: F2D2E977F7584616B2D6EFA3D04FC993
2016-05-18 00:58:05,484 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_ALL"]
2016-05-18 00:58:05,484 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:05,485 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E479C8CEB95643F48BE893CE33DC276B, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:05,487 INFO output.MROutput (MROutput.java:close(526)) - F2D2E977F7584616B2D6EFA3D04FC993 closed
2016-05-18 00:58:05,488 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533085267_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:05,489 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533085267_0001_1_00_000000_0
2016-05-18 00:58:05,489 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E479C8CEB95643F48BE893CE33DC276B, mem on close (mb), free: 204, total: 378, max: 672
2016-05-18 00:58:05,489 INFO output.MROutput (MROutput.java:close(526)) - F2D2E977F7584616B2D6EFA3D04FC993 closed
2016-05-18 00:58:05,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:05 Completed running task attempt: attempt_1463533085267_0001_1_00_000000_0
2016-05-18 00:58:05,490 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533085267_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:05,490 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533085267_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:05,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533085267_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533085485, Process_Duration=4, Process_End_Time=1463533085489, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:05,492 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533085267_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:05,493 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:05,493 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:05,493 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:05,493 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533085267_0001_00_000001
2016-05-18 00:58:05,494 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085267_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E479C8CEB95643F48BE893CE33DC276B, taskAttemptId=attempt_1463533085267_0001_1_00_000000_0, creationTime=1463533085450, allocationTime=1463533085451, startTime=1463533085453, finishTime=1463533085492, timeTaken=39, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:05,495 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533085267_0001_1_00_000000_0
2016-05-18 00:58:05,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085267_0001_1][Event:TASK_FINISHED]: vertexName=E479C8CEB95643F48BE893CE33DC276B, taskId=task_1463533085267_0001_1_00_000000, startTime=1463533085453, finishTime=1463533085497, timeTaken=44, status=SUCCEEDED, successfulAttemptID=attempt_1463533085267_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533085485, Process_Duration=4, Process_End_Time=1463533085489, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:05,498 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:05,498 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B]
2016-05-18 00:58:05,499 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085267_0001_1][Event:VERTEX_FINISHED]: vertexName=E479C8CEB95643F48BE893CE33DC276B, vertexId=vertex_1463533085267_0001_1_00, initRequestedTime=1463533085433, initedTime=1463533085444, startRequestedTime=1463533085437, startedTime=1463533085447, finishTime=1463533085498, timeTaken=51, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533085485, Process_Duration=4, Process_End_Time=1463533085489, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533085453, firstTasksToStart=[ task_1463533085267_0001_1_00_000000 ], lastTaskFinishTime=1463533085492, lastTasksToFinish=[ task_1463533085267_0001_1_00_000000 ], minTaskDuration=39, maxTaskDuration=39, avgTaskDuration=39.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533085267_0001_1_00_000000 ], longestDurationTasks=[ task_1463533085267_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:05,500 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:05,500 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085267_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533085267_0001_00_000001, stoppedTime=1463533085500, exitStatus=0
2016-05-18 00:58:05,500 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:05,500 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:05,500 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533085267_0001_1
2016-05-18 00:58:05,500 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085267_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533085267_0001_1
2016-05-18 00:58:05,501 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: F2D2E977F7584616B2D6EFA3D04FC993 for vertex: vertex_1463533085267_0001_1_00 [E479C8CEB95643F48BE893CE33DC276B], outputName: F2D2E977F7584616B2D6EFA3D04FC993
2016-05-18 00:58:05,501 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533085267_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:05,501 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533085267_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:05,507 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:F2D2E977F7584616B2D6EFA3D04FC993 of vertex/vertexGroup:E479C8CEB95643F48BE893CE33DC276B isVertexGroupOutput:false
2016-05-18 00:58:05,507 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:05,507 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533085267_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:05,507 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533085267_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:05,508 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533085267_0001_00_000001, Exiting
2016-05-18 00:58:05,508 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533085267_0001_00_000001
2016-05-18 00:58:05,508 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533085267_0001_00_000001
2016-05-18 00:58:05,508 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533085267_0001_1, queueSize=0
2016-05-18 00:58:05,508 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533085267_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:05,509 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085267_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533085267_0001_1, startTime=1463533085433, finishTime=1463533085507, timeTaken=74, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=130, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533085485, Process_Duration=4, Process_End_Time=1463533085489, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:05,509 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533085267_0001_00_000001 completed successfully
2016-05-18 00:58:05,510 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533085267_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:05,510 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533085267_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:05 Completed Dag: dag_1463533085267_0001_1
2016-05-18 00:58:05,510 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533085267_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:05,510 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533085267_0001_1_post
2016-05-18 00:58:05,511 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:05,511 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 1
2016-05-18 00:58:05,511 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:05,511 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[6F6264F24CAA4195958E2C669CFA6FA7/A0A71F48AFEE40FDB909B397FECEB029] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL, with id=dag_1463533085267_0001_1
2016-05-18 00:58:05,511 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[6F6264F24CAA4195958E2C669CFA6FA7/A0A71F48AFEE40FDB909B397FECEB029] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL, with id=dag_1463533085267_0001_1
2016-05-18 00:58:05,520 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL, applicationId=application_1463533085267_0001
2016-05-18 00:58:05,522 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:05,522 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:05,522 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:05,522 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:05,522 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:05,523 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:05,523 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:05,524 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:05,524 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:05,524 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:05,525 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:05,525 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:05,526 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:05,526 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:05,529 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/6F6264F24CAA4195958E2C669CFA6FA7/A0A71F48AFEE40FDB909B397FECEB029/.tez/application_1463533085267_0001
2016-05-18 00:58:05,530 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_ALL/_temporary
2016-05-18 00:58:05,530 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.285
2016-05-18 00:58:05,604 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_UNKNOWN_ALL
2016-05-18 00:58:05,629 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:05,629 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:05,676 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-05-18 00:58:05,680 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-05-18 00:58:05,681 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:05,681 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:05,681 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:05,697 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:05,698 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_-1
2016-05-18 00:58:05,709 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:05,709 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:05,709 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_-1"]
2016-05-18 00:58:05,710 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:05,710 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:05,710 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:05,712 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-18 00:58:05,712 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:05,726 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:05,730 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:05,730 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:05,731 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:05,739 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/AE26347EFE3846548C956C9A2242D3DD/EBC453B147E64286BF50F291C1537A0C/.tez/application_1463533085730_0001 doesn't exist and is created
2016-05-18 00:58:05,754 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:05,754 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:05,755 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/AE26347EFE3846548C956C9A2242D3DD/EBC453B147E64286BF50F291C1537A0C/.tez/application_1463533085730_0001_wd
2016-05-18 00:58:05,766 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533085730_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:05,766 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:05,766 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:05,768 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:05,768 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:05,768 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:05,768 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:05,769 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:05,771 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:05,771 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:05,771 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:05,771 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:05,772 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:05,772 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:05,772 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:05,772 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:05,773 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:05,773 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:05,773 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:05,774 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:05,776 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:05,777 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:05,777 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:05,777 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/AE26347EFE3846548C956C9A2242D3DD/EBC453B147E64286BF50F291C1537A0C/.tez/application_1463533085730_0001_wd/localmode-log-dir
2016-05-18 00:58:05,778 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/AE26347EFE3846548C956C9A2242D3DD/EBC453B147E64286BF50F291C1537A0C/.tez/application_1463533085730_0001_wd/localmode-log-dir/history.txt.appattempt_1463533085730_0001_000000, maxErrors=10
2016-05-18 00:58:05,778 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/AE26347EFE3846548C956C9A2242D3DD/EBC453B147E64286BF50F291C1537A0C/.tez/application_1463533085730_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:05,778 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533085730_0001_000000, appSubmitTime=1463533085765, launchTime=1463533085765
2016-05-18 00:58:05,782 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:05,787 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:54860
2016-05-18 00:58:05,787 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533085730
2016-05-18 00:58:05,790 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533085730_0001_000000, startTime=1463533085789
2016-05-18 00:58:05,790 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:05,830 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:58:05,830 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:58:05,854 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:05,855 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:05,855 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.-1_VALUES_-1, applicationId=application_1463533085730_0001, dagName=[AE26347EFE3846548C956C9A2242D3DD/EBC453B147E64286BF50F291C1537A0C] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-18 00:58:05,903 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [AE26347EFE3846548C956C9A2242D3DD/EBC453B147E64286BF50F291C1537A0C] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-18 00:58:05,903 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:05,904 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533085730_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/AE26347EFE3846548C956C9A2242D3DD/EBC453B147E64286BF50F291C1537A0C/.tez/application_1463533085730_0001_wd/localmode-log-dir/dag_1463533085730_0001_1.dot
2016-05-18 00:58:05,905 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/AE26347EFE3846548C956C9A2242D3DD/EBC453B147E64286BF50F291C1537A0C/.tez/application_1463533085730_0001_wd/localmode-log-dir/dag_1463533085730_0001_1-tez-dag.pb.txt
2016-05-18 00:58:05,909 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533085730_0001_1
2016-05-18 00:58:05,909 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:05,909 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [AE26347EFE3846548C956C9A2242D3DD/EBC453B147E64286BF50F291C1537A0C] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-18 00:58:05 Running Dag: dag_1463533085730_0001_1
2016-05-18 00:58:05,919 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085730_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533085730_0001_1, submitTime=1463533085903
2016-05-18 00:58:05,921 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/AE26347EFE3846548C956C9A2242D3DD/EBC453B147E64286BF50F291C1537A0C/.tez/application_1463533085730_0001/tez-conf.pb" } size: 36913 timestamp: 1463533085000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:05,921 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF]
2016-05-18 00:58:05,922 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF]
2016-05-18 00:58:05,922 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:05,922 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:05,923 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085730_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533085730_0001_1, initTime=1463533085922
2016-05-18 00:58:05,924 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533085730_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:05,924 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:05,925 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085730_0001_1][Event:DAG_STARTED]: dagID=dag_1463533085730_0001_1, startTime=1463533085925
2016-05-18 00:58:05,925 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533085730_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:05,925 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: E0F2C487925D4C609824CB4DC6FCA8FF : {61A42AF168714CFEAFEBB5D70B7DF1E6={InputName=61A42AF168714CFEAFEBB5D70B7DF1E6}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:05,925 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 61A42AF168714CFEAFEBB5D70B7DF1E6, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:05,925 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF]
2016-05-18 00:58:05,925 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF]
2016-05-18 00:58:05,926 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF]
2016-05-18 00:58:05,927 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF]
2016-05-18 00:58:05,926 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.-1_VALUES_-1, applicationId=application_1463533085730_0001, dagName=[AE26347EFE3846548C956C9A2242D3DD/EBC453B147E64286BF50F291C1537A0C] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-18 00:58:05,927 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:05,927 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 61A42AF168714CFEAFEBB5D70B7DF1E6 on vertex vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF]
2016-05-18 00:58:05,930 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 61A42AF168714CFEAFEBB5D70B7DF1E6 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:05,931 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:05,932 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:05,932 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:05,932 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:05,932 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 61A42AF168714CFEAFEBB5D70B7DF1E6 on vertex vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF]
2016-05-18 00:58:05,933 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:05,935 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {61A42AF168714CFEAFEBB5D70B7DF1E6=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:05,936 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF] parallelism set to 1
2016-05-18 00:58:05,936 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:05,937 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF], numAdditionalOutputs=1
2016-05-18 00:58:05,937 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=FBAE61A80084452B8F60D748F02A6CB8, vertex=vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:05,938 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for E0F2C487925D4C609824CB4DC6FCA8FF:FBAE61A80084452B8F60D748F02A6CB8 using oldmapred API
2016-05-18 00:58:05,939 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=FBAE61A80084452B8F60D748F02A6CB8, vertexName=E0F2C487925D4C609824CB4DC6FCA8FF, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:05,940 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085730_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E0F2C487925D4C609824CB4DC6FCA8FF, vertexId=vertex_1463533085730_0001_1_00, initRequestedTime=1463533085925, initedTime=1463533085937, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:05,940 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF] with distanceFromRoot: 0
2016-05-18 00:58:05,940 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:05,940 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085730_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533085730_0001_1_00, startRequestedTime=1463533085927, startedTime=1463533085940
2016-05-18 00:58:05,940 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085730_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533085730_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:05,940 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:05,940 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in E0F2C487925D4C609824CB4DC6FCA8FF
2016-05-18 00:58:05,941 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF]
2016-05-18 00:58:05,941 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085730_0001_1][Event:TASK_STARTED]: vertexName=E0F2C487925D4C609824CB4DC6FCA8FF, taskId=task_1463533085730_0001_1_00_000000, scheduledTime=1463533085941, launchTime=1463533085941
2016-05-18 00:58:05,942 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF]
2016-05-18 00:58:05,943 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:05,945 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533085730_0001_00_000001, launchTime=1463533085944
2016-05-18 00:58:05,945 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533085730_0001_00_000001
2016-05-18 00:58:05,946 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533085730_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533085730_0001_1_00_000000_0
2016-05-18 00:58:05,946 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533085730_0001_1_00_000000_0] started. Is using containerId: [container_1463533085730_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:05,946 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085730_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E0F2C487925D4C609824CB4DC6FCA8FF, taskAttemptId=attempt_1463533085730_0001_1_00_000000_0, startTime=1463533085946, containerId=container_1463533085730_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:05 Starting to run new task attempt: attempt_1463533085730_0001_1_00_000000_0
2016-05-18 00:58:05,947 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533085730_0001_1_00_000000_0
2016-05-18 00:58:05,947 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:05,947 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:05,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [AE26347EFE3846548C956C9A2242D3DD/EBC453B147E64286BF50F291C1537A0C] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1, VertexName: E0F2C487925D4C609824CB4DC6FCA8FF, VertexParallelism: 1, TaskAttemptID:attempt_1463533085730_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=61A42AF168714CFEAFEBB5D70B7DF1E6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FBAE61A80084452B8F60D748F02A6CB8, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:05,948 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:05,948 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533085730_0001_1_00_000000_0
2016-05-18 00:58:05,949 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533085730_0001, with dag id: dag_1463533085730_0001_1
2016-05-18 00:58:05,950 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533085730_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:05,951 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533085730_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:05,957 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:05,958 INFO input.MRInput (MRInput.java:initialize(440)) - 61A42AF168714CFEAFEBB5D70B7DF1E6 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:05,958 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 61A42AF168714CFEAFEBB5D70B7DF1E6
2016-05-18 00:58:05,966 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E0F2C487925D4C609824CB4DC6FCA8FF, ordinal: 0
2016-05-18 00:58:05,966 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E0F2C487925D4C609824CB4DC6FCA8FF, mem on start (mb), free: 303, total: 374, max: 672
2016-05-18 00:58:05,966 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:05,966 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:05,967 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:05,967 INFO output.MROutput (MROutput.java:initialize(421)) - FBAE61A80084452B8F60D748F02A6CB8: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:05,968 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:05,969 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[61A42AF168714CFEAFEBB5D70B7DF1E6:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [FBAE61A80084452B8F60D748F02A6CB8:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:05,969 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:05,969 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:05,969 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:05,969 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[61A42AF168714CFEAFEBB5D70B7DF1E6:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [FBAE61A80084452B8F60D748F02A6CB8:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:05,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:05,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 61A42AF168714CFEAFEBB5D70B7DF1E6
2016-05-18 00:58:05,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 61A42AF168714CFEAFEBB5D70B7DF1E6 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:05,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:05,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:05,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:05,971 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533085730_0001_1_00_000000_0
2016-05-18 00:58:05,974 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:05,975 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:05,975 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 61A42AF168714CFEAFEBB5D70B7DF1E6 initialized RecordReader from event
2016-05-18 00:58:05,975 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 61A42AF168714CFEAFEBB5D70B7DF1E6
2016-05-18 00:58:05,976 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_-1"], id: FBAE61A80084452B8F60D748F02A6CB8
2016-05-18 00:58:05,976 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_-1"]
2016-05-18 00:58:05,976 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:05,976 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E0F2C487925D4C609824CB4DC6FCA8FF, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:05,978 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 00:58:05,984 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 00:58:05,988 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 00:58:05,989 INFO output.MROutput (MROutput.java:close(526)) - FBAE61A80084452B8F60D748F02A6CB8 closed
2016-05-18 00:58:05,990 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533085730_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:05,991 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533085730_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:05,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533085730_0001_1_00_000000_0: Counters: 8 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463533085977, Process_Duration=14, Process_End_Time=1463533085991, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-18 00:58:05,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(817)) - Joining on EventRouter
2016-05-18 00:58:05,992 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E0F2C487925D4C609824CB4DC6FCA8FF, mem on close (mb), free: 301, total: 374, max: 672
2016-05-18 00:58:05,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(851)) - Closed processor for vertex=E0F2C487925D4C609824CB4DC6FCA8FF, index=0, interruptedStatus=false
2016-05-18 00:58:05,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(887)) - Closed input for vertex=E0F2C487925D4C609824CB4DC6FCA8FF, sourceVertex=61A42AF168714CFEAFEBB5D70B7DF1E6, interruptedStatus=true
2016-05-18 00:58:05,993 INFO output.MROutput (MROutput.java:close(526)) - FBAE61A80084452B8F60D748F02A6CB8 closed
2016-05-18 00:58:05,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(911)) - Closed input for vertex=E0F2C487925D4C609824CB4DC6FCA8FF, sourceVertex=FBAE61A80084452B8F60D748F02A6CB8, interruptedStatus=true
2016-05-18 00:58:05,996 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533085730_0001_1_00_000000_0 : TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 00:58:05,997 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 00:58:05,997 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533085730_0001_00_000001
2016-05-18 00:58:05,997 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533085730_0001_00_000001
2016-05-18 00:58:06,001 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533085730_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, errorMessage='TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation'}
2016-05-18 00:58:06,001 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(294)) - Container: container_1463533085730_0001_00_000001 completed but with errors
2016-05-18 00:58:06,001 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085730_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E0F2C487925D4C609824CB4DC6FCA8FF, taskAttemptId=attempt_1463533085730_0001_1_00_000000_0, creationTime=1463533085941, allocationTime=1463533085944, startTime=1463533085946, finishTime=1463533085996, timeTaken=50, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task: attempt_1463533085730_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
, counters=Counters: 9, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463533085977, Process_Duration=14, Process_End_Time=1463533085991, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-18 00:58:06,004 INFO container.AMContainerImpl (AMContainerImpl.java:transition(704)) - Container container_1463533085730_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
2016-05-18 00:58:06,004 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085730_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533085730_0001_00_000001, stoppedTime=1463533086004, exitStatus=1
2016-05-18 00:58:06,005 INFO impl.TaskImpl (TaskImpl.java:transition(1170)) - Failing task: task_1463533085730_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-18 00:58:06,005 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085730_0001_1][Event:TASK_FINISHED]: vertexName=E0F2C487925D4C609824CB4DC6FCA8FF, taskId=task_1463533085730_0001_1_00_000000, startTime=1463533085946, finishTime=1463533086005, timeTaken=59, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463533085730_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
], counters=Counters: 0
2016-05-18 00:58:06,007 INFO impl.VertexImpl (VertexImpl.java:transition(3353)) - Failing vertex: vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF] because task failed: task_1463533085730_0001_1_00_000000
2016-05-18 00:58:06,007 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(2102)) - Killing tasks in vertex: vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF] due to trigger: OWN_TASK_FAILURE
2016-05-18 00:58:06,008 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF], tasks=1, failed=1, killed=0, success=0, completed=1, commits=0, err=OWN_TASK_FAILURE
2016-05-18 00:58:06,008 INFO impl.VertexImpl (VertexImpl.java:finishWithTerminationCause(2084)) - Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
2016-05-18 00:58:06,009 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3087)) - Invoking committer abort for vertex, vertexId=vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF]
2016-05-18 00:58:06,010 INFO impl.VertexImpl (VertexImpl.java:run(3095)) - Invoking committer abort for output=FBAE61A80084452B8F60D748F02A6CB8, vertexId=vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF]
2016-05-18 00:58:06,016 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085730_0001_1][Event:VERTEX_FINISHED]: vertexName=E0F2C487925D4C609824CB4DC6FCA8FF, vertexId=vertex_1463533085730_0001_1_00, initRequestedTime=1463533085925, initedTime=1463533085937, startRequestedTime=1463533085927, startedTime=1463533085940, finishTime=1463533086014, timeTaken=74, status=FAILED, diagnostics=Task failed, taskId=task_1463533085730_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463533085730_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]]
Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
Vertex vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF] killed/failed due to:OWN_TASK_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=1463533085946, firstTasksToStart=[ task_1463533085730_0001_1_00_000000 ], lastTaskFinishTime=1463533085996, lastTasksToFinish=[ task_1463533085730_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-18 00:58:06,016 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-18 00:58:06,017 INFO node.AMNodeImpl (AMNodeImpl.java:transition(298)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463533085730_0001_1_00_000000_0 failed: true container: container_1463533085730_0001_00_000001 numFailedTAs: 0
2016-05-18 00:58:06,018 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-18 00:58:06,019 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=VERTEX_FAILURE
2016-05-18 00:58:06,019 INFO impl.DAGImpl (DAGImpl.java:finishWithTerminationCause(1350)) - DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
2016-05-18 00:58:06,020 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533085730_0001_1, queueSize=0
2016-05-18 00:58:06,021 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533085730_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533085730_0001_1, startTime=1463533085924, finishTime=1463533086020, timeTaken=96, status=FAILED, diagnostics=Vertex failed, vertexName=E0F2C487925D4C609824CB4DC6FCA8FF, vertexId=vertex_1463533085730_0001_1_00, diagnostics=[Task failed, taskId=task_1463533085730_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463533085730_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF] killed/failed due to:OWN_TASK_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0, counters=Counters: 4, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, AM_CPU_MILLISECONDS=180, AM_GC_TIME_MILLIS=0
2016-05-18 00:58:06,021 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3082)) - Ignoring multiple aborts for vertex: vertex_1463533085730_0001_1_00 [E0F2C487925D4C609824CB4DC6FCA8FF]
2016-05-18 00:58:06,021 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533085730_0001_1 finished with state: FAILED
2016-05-18 00:58:06,021 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533085730_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:06 Completed Dag: dag_1463533085730_0001_1
2016-05-18 00:58:06,021 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533085730_0001_1, dagState=FAILED
2016-05-18 00:58:06,022 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533085730_0001_1_post
2016-05-18 00:58:06,022 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:06,022 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:06,022 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:06,022 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[AE26347EFE3846548C956C9A2242D3DD/EBC453B147E64286BF50F291C1537A0C] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1, with id=dag_1463533085730_0001_1
2016-05-18 00:58:06,022 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[AE26347EFE3846548C956C9A2242D3DD/EBC453B147E64286BF50F291C1537A0C] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1, with id=dag_1463533085730_0001_1
2016-05-18 00:58:06,031 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...eld'_RESULTS.-1_VALUES_-1, applicationId=application_1463533085730_0001
2016-05-18 00:58:06,034 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:06,034 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:06,034 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:06,035 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:06,035 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:06,035 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:06,035 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:06,037 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:06,037 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:06,037 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:06,038 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:06,038 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:06,038 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:06,039 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:06,042 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/AE26347EFE3846548C956C9A2242D3DD/EBC453B147E64286BF50F291C1537A0C/.tez/application_1463533085730_0001
2016-05-18 00:58:06,043 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-18 00:58:06,043 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-18 00:58:06,045 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-18 00:58:06,045 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_-1/_temporary
2016-05-18 00:58:06,046 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.336
2016-05-18 00:58:06,046 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_VALUES_-1
2016-05-18 00:58:06,075 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:06,075 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:06,111 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-05-18 00:58:06,114 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.008
2016-05-18 00:58:06,114 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 00:58:06,115 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 00:58:06,115 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_VALUES_'field2'
2016-05-18 00:58:06,144 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:06,144 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:06,187 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-18 00:58:06,195 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-05-18 00:58:06,196 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:06,196 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:06,196 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:06,213 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:06,214 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_RESULTS
2016-05-18 00:58:06,226 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:06,227 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:06,227 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_RESULTS"]
2016-05-18 00:58:06,227 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:06,228 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:06,228 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:06,229 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-18 00:58:06,229 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:06,244 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:06,248 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:06,249 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:06,249 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:06,258 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/764402181712400182BE4B6850478F3E/E6F7C90B14AD49C4816B6EA4BE11752C/.tez/application_1463533086249_0001 doesn't exist and is created
2016-05-18 00:58:06,272 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:06,272 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:06,273 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/764402181712400182BE4B6850478F3E/E6F7C90B14AD49C4816B6EA4BE11752C/.tez/application_1463533086249_0001_wd
2016-05-18 00:58:06,285 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533086249_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:06,285 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:06,285 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:06,286 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:06,287 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:06,287 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:06,287 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:06,288 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:06,289 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:06,289 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:06,289 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:06,290 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:06,290 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:06,290 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:06,291 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:06,291 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:06,292 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:06,293 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:06,293 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:06,294 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:06,297 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:06,298 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:06,298 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:06,299 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/764402181712400182BE4B6850478F3E/E6F7C90B14AD49C4816B6EA4BE11752C/.tez/application_1463533086249_0001_wd/localmode-log-dir
2016-05-18 00:58:06,299 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/764402181712400182BE4B6850478F3E/E6F7C90B14AD49C4816B6EA4BE11752C/.tez/application_1463533086249_0001_wd/localmode-log-dir/history.txt.appattempt_1463533086249_0001_000000, maxErrors=10
2016-05-18 00:58:06,299 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/764402181712400182BE4B6850478F3E/E6F7C90B14AD49C4816B6EA4BE11752C/.tez/application_1463533086249_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:06,299 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533086249_0001_000000, appSubmitTime=1463533086284, launchTime=1463533086284
2016-05-18 00:58:06,307 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:06,310 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:41762
2016-05-18 00:58:06,310 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533086249
2016-05-18 00:58:06,314 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533086249_0001_000000, startTime=1463533086314
2016-05-18 00:58:06,314 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:06,373 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:06,373 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:06,373 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...RESULTS.-1_VALUES_RESULTS, applicationId=application_1463533086249_0001, dagName=[764402181712400182BE4B6850478F3E/E6F7C90B14AD49C4816B6EA4BE11752C] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-18 00:58:06,396 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [764402181712400182BE4B6850478F3E/E6F7C90B14AD49C4816B6EA4BE11752C] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-18 00:58:06,396 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:06,397 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533086249_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/764402181712400182BE4B6850478F3E/E6F7C90B14AD49C4816B6EA4BE11752C/.tez/application_1463533086249_0001_wd/localmode-log-dir/dag_1463533086249_0001_1.dot
2016-05-18 00:58:06,397 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/764402181712400182BE4B6850478F3E/E6F7C90B14AD49C4816B6EA4BE11752C/.tez/application_1463533086249_0001_wd/localmode-log-dir/dag_1463533086249_0001_1-tez-dag.pb.txt
2016-05-18 00:58:06,401 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533086249_0001_1
2016-05-18 00:58:06,401 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:06,402 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [764402181712400182BE4B6850478F3E/E6F7C90B14AD49C4816B6EA4BE11752C] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-18 00:58:06 Running Dag: dag_1463533086249_0001_1
2016-05-18 00:58:06,411 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086249_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533086249_0001_1, submitTime=1463533086396
2016-05-18 00:58:06,412 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/764402181712400182BE4B6850478F3E/E6F7C90B14AD49C4816B6EA4BE11752C/.tez/application_1463533086249_0001/tez-conf.pb" } size: 36913 timestamp: 1463533086000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:06,413 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4]
2016-05-18 00:58:06,413 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4]
2016-05-18 00:58:06,414 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:06,414 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:06,415 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086249_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533086249_0001_1, initTime=1463533086415
2016-05-18 00:58:06,416 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533086249_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:06,416 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:06,416 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086249_0001_1][Event:DAG_STARTED]: dagID=dag_1463533086249_0001_1, startTime=1463533086416
2016-05-18 00:58:06,417 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533086249_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:06,417 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: EC32A2061BEA4DD59F2D59C950DD8DA4 : {8A39B83D85764801BFB93DFDCC0C2E34={InputName=8A39B83D85764801BFB93DFDCC0C2E34}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:06,417 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 8A39B83D85764801BFB93DFDCC0C2E34, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:06,417 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4]
2016-05-18 00:58:06,417 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4]
2016-05-18 00:58:06,417 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4]
2016-05-18 00:58:06,418 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4]
2016-05-18 00:58:06,418 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...RESULTS.-1_VALUES_RESULTS, applicationId=application_1463533086249_0001, dagName=[764402181712400182BE4B6850478F3E/E6F7C90B14AD49C4816B6EA4BE11752C] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-18 00:58:06,418 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:06,419 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 8A39B83D85764801BFB93DFDCC0C2E34 on vertex vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4]
2016-05-18 00:58:06,421 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 8A39B83D85764801BFB93DFDCC0C2E34 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:06,423 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:06,423 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:06,423 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:06,424 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:06,424 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 8A39B83D85764801BFB93DFDCC0C2E34 on vertex vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4]
2016-05-18 00:58:06,425 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:06,426 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {8A39B83D85764801BFB93DFDCC0C2E34=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:06,427 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4] parallelism set to 1
2016-05-18 00:58:06,427 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:06,427 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4], numAdditionalOutputs=1
2016-05-18 00:58:06,427 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=2551BC3DF42042A8A5306A9CF3C59D0C, vertex=vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:06,429 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for EC32A2061BEA4DD59F2D59C950DD8DA4:2551BC3DF42042A8A5306A9CF3C59D0C using oldmapred API
2016-05-18 00:58:06,429 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=2551BC3DF42042A8A5306A9CF3C59D0C, vertexName=EC32A2061BEA4DD59F2D59C950DD8DA4, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:06,430 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086249_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EC32A2061BEA4DD59F2D59C950DD8DA4, vertexId=vertex_1463533086249_0001_1_00, initRequestedTime=1463533086417, initedTime=1463533086427, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:06,431 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4] with distanceFromRoot: 0
2016-05-18 00:58:06,431 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:06,431 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086249_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533086249_0001_1_00, startRequestedTime=1463533086418, startedTime=1463533086431
2016-05-18 00:58:06,431 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086249_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533086249_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:06,431 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:06,432 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in EC32A2061BEA4DD59F2D59C950DD8DA4
2016-05-18 00:58:06,432 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4]
2016-05-18 00:58:06,434 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086249_0001_1][Event:TASK_STARTED]: vertexName=EC32A2061BEA4DD59F2D59C950DD8DA4, taskId=task_1463533086249_0001_1_00_000000, scheduledTime=1463533086434, launchTime=1463533086434
2016-05-18 00:58:06,434 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4]
2016-05-18 00:58:06,435 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:06,437 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533086249_0001_00_000001, launchTime=1463533086437
2016-05-18 00:58:06,437 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533086249_0001_00_000001
2016-05-18 00:58:06,438 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533086249_0001_1_00_000000_0] started. Is using containerId: [container_1463533086249_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:06,439 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086249_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EC32A2061BEA4DD59F2D59C950DD8DA4, taskAttemptId=attempt_1463533086249_0001_1_00_000000_0, startTime=1463533086438, containerId=container_1463533086249_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:06,440 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533086249_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533086249_0001_1_00_000000_0
2016-05-18 00:58:06 Starting to run new task attempt: attempt_1463533086249_0001_1_00_000000_0
2016-05-18 00:58:06,441 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533086249_0001_1_00_000000_0
2016-05-18 00:58:06,441 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:06,441 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:06,441 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [764402181712400182BE4B6850478F3E/E6F7C90B14AD49C4816B6EA4BE11752C] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS, VertexName: EC32A2061BEA4DD59F2D59C950DD8DA4, VertexParallelism: 1, TaskAttemptID:attempt_1463533086249_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8A39B83D85764801BFB93DFDCC0C2E34, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=2551BC3DF42042A8A5306A9CF3C59D0C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:06,441 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:06,443 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533086249_0001_1_00_000000_0
2016-05-18 00:58:06,445 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533086249_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:06,445 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533086249_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:06,446 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533086249_0001, with dag id: dag_1463533086249_0001_1
2016-05-18 00:58:06,450 INFO input.MRInput (MRInput.java:initialize(440)) - 8A39B83D85764801BFB93DFDCC0C2E34 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:06,451 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 8A39B83D85764801BFB93DFDCC0C2E34
2016-05-18 00:58:06,451 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:06,461 INFO output.MROutput (MROutput.java:initialize(421)) - 2551BC3DF42042A8A5306A9CF3C59D0C: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:06,464 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EC32A2061BEA4DD59F2D59C950DD8DA4, ordinal: 0
2016-05-18 00:58:06,464 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EC32A2061BEA4DD59F2D59C950DD8DA4, mem on start (mb), free: 213, total: 374, max: 672
2016-05-18 00:58:06,464 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:06,464 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:06,465 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:06,465 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:06,465 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[8A39B83D85764801BFB93DFDCC0C2E34:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [2551BC3DF42042A8A5306A9CF3C59D0C:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:06,466 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:06,467 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:06,467 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:06,467 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[8A39B83D85764801BFB93DFDCC0C2E34:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [2551BC3DF42042A8A5306A9CF3C59D0C:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:06,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:06,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 8A39B83D85764801BFB93DFDCC0C2E34
2016-05-18 00:58:06,468 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 8A39B83D85764801BFB93DFDCC0C2E34 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:06,468 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:06,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:06,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:06,469 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533086249_0001_1_00_000000_0
2016-05-18 00:58:06,472 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:06,472 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 8A39B83D85764801BFB93DFDCC0C2E34 initialized RecordReader from event
2016-05-18 00:58:06,473 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:06,473 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 8A39B83D85764801BFB93DFDCC0C2E34
2016-05-18 00:58:06,474 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_RESULTS"], id: 2551BC3DF42042A8A5306A9CF3C59D0C
2016-05-18 00:58:06,474 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_RESULTS"]
2016-05-18 00:58:06,474 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:06,475 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EC32A2061BEA4DD59F2D59C950DD8DA4, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:06,476 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 00:58:06,478 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 00:58:06,478 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 00:58:06,480 INFO output.MROutput (MROutput.java:close(526)) - 2551BC3DF42042A8A5306A9CF3C59D0C closed
2016-05-18 00:58:06,480 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533086249_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:06,481 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533086249_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:06,482 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533086249_0001_1_00_000000_0: Counters: 8 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463533086475, Process_Duration=6, Process_End_Time=1463533086481, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-18 00:58:06,482 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(817)) - Joining on EventRouter
2016-05-18 00:58:06,482 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EC32A2061BEA4DD59F2D59C950DD8DA4, mem on close (mb), free: 211, total: 374, max: 672
2016-05-18 00:58:06,483 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(851)) - Closed processor for vertex=EC32A2061BEA4DD59F2D59C950DD8DA4, index=0, interruptedStatus=false
2016-05-18 00:58:06,483 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(887)) - Closed input for vertex=EC32A2061BEA4DD59F2D59C950DD8DA4, sourceVertex=8A39B83D85764801BFB93DFDCC0C2E34, interruptedStatus=true
2016-05-18 00:58:06,483 INFO output.MROutput (MROutput.java:close(526)) - 2551BC3DF42042A8A5306A9CF3C59D0C closed
2016-05-18 00:58:06,484 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(911)) - Closed input for vertex=EC32A2061BEA4DD59F2D59C950DD8DA4, sourceVertex=2551BC3DF42042A8A5306A9CF3C59D0C, interruptedStatus=true
2016-05-18 00:58:06,485 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533086249_0001_1_00_000000_0 : TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 00:58:06,485 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 00:58:06,485 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533086249_0001_00_000001
2016-05-18 00:58:06,485 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086249_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EC32A2061BEA4DD59F2D59C950DD8DA4, taskAttemptId=attempt_1463533086249_0001_1_00_000000_0, creationTime=1463533086434, allocationTime=1463533086436, startTime=1463533086438, finishTime=1463533086485, timeTaken=47, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task: attempt_1463533086249_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
, counters=Counters: 9, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463533086475, Process_Duration=6, Process_End_Time=1463533086481, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-18 00:58:06,485 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533086249_0001_00_000001
2016-05-18 00:58:06,486 INFO impl.TaskImpl (TaskImpl.java:transition(1170)) - Failing task: task_1463533086249_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-18 00:58:06,486 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086249_0001_1][Event:TASK_FINISHED]: vertexName=EC32A2061BEA4DD59F2D59C950DD8DA4, taskId=task_1463533086249_0001_1_00_000000, startTime=1463533086438, finishTime=1463533086486, timeTaken=48, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463533086249_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
], counters=Counters: 0
2016-05-18 00:58:06,486 INFO impl.VertexImpl (VertexImpl.java:transition(3353)) - Failing vertex: vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4] because task failed: task_1463533086249_0001_1_00_000000
2016-05-18 00:58:06,487 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(2102)) - Killing tasks in vertex: vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4] due to trigger: OWN_TASK_FAILURE
2016-05-18 00:58:06,487 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4], tasks=1, failed=1, killed=0, success=0, completed=1, commits=0, err=OWN_TASK_FAILURE
2016-05-18 00:58:06,487 INFO impl.VertexImpl (VertexImpl.java:finishWithTerminationCause(2084)) - Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
2016-05-18 00:58:06,487 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3087)) - Invoking committer abort for vertex, vertexId=vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4]
2016-05-18 00:58:06,487 INFO impl.VertexImpl (VertexImpl.java:run(3095)) - Invoking committer abort for output=2551BC3DF42042A8A5306A9CF3C59D0C, vertexId=vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4]
2016-05-18 00:58:06,492 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086249_0001_1][Event:VERTEX_FINISHED]: vertexName=EC32A2061BEA4DD59F2D59C950DD8DA4, vertexId=vertex_1463533086249_0001_1_00, initRequestedTime=1463533086417, initedTime=1463533086427, startRequestedTime=1463533086418, startedTime=1463533086431, finishTime=1463533086489, timeTaken=58, status=FAILED, diagnostics=Task failed, taskId=task_1463533086249_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463533086249_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]]
Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
Vertex vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4] killed/failed due to:OWN_TASK_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=1463533086438, firstTasksToStart=[ task_1463533086249_0001_1_00_000000 ], lastTaskFinishTime=1463533086485, lastTasksToFinish=[ task_1463533086249_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-18 00:58:06,492 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-18 00:58:06,492 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-18 00:58:06,492 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=VERTEX_FAILURE
2016-05-18 00:58:06,492 INFO impl.DAGImpl (DAGImpl.java:finishWithTerminationCause(1350)) - DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
2016-05-18 00:58:06,492 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533086249_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, errorMessage='TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation'}
2016-05-18 00:58:06,493 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(294)) - Container: container_1463533086249_0001_00_000001 completed but with errors
2016-05-18 00:58:06,494 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533086249_0001_1, queueSize=0
2016-05-18 00:58:06,495 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086249_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533086249_0001_1, startTime=1463533086416, finishTime=1463533086493, timeTaken=77, status=FAILED, diagnostics=Vertex failed, vertexName=EC32A2061BEA4DD59F2D59C950DD8DA4, vertexId=vertex_1463533086249_0001_1_00, diagnostics=[Task failed, taskId=task_1463533086249_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463533086249_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4] killed/failed due to:OWN_TASK_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0, counters=Counters: 4, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, AM_CPU_MILLISECONDS=160, AM_GC_TIME_MILLIS=0
2016-05-18 00:58:06,495 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3082)) - Ignoring multiple aborts for vertex: vertex_1463533086249_0001_1_00 [EC32A2061BEA4DD59F2D59C950DD8DA4]
2016-05-18 00:58:06,496 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533086249_0001_1 finished with state: FAILED
2016-05-18 00:58:06,496 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533086249_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:06,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086249_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533086249_0001_00_000001, stoppedTime=1463533086497, exitStatus=-1000
2016-05-18 00:58:06,497 INFO node.AMNodeImpl (AMNodeImpl.java:transition(298)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463533086249_0001_1_00_000000_0 failed: true container: container_1463533086249_0001_00_000001 numFailedTAs: 0
2016-05-18 00:58:06,498 INFO container.AMContainerImpl (AMContainerImpl.java:transition(993)) - Container container_1463533086249_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
2016-05-18 00:58:06 Completed Dag: dag_1463533086249_0001_1
2016-05-18 00:58:06,502 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533086249_0001_1, dagState=FAILED
2016-05-18 00:58:06,502 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533086249_0001_1_post
2016-05-18 00:58:06,502 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:06,502 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 3
2016-05-18 00:58:06,502 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:06,502 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[764402181712400182BE4B6850478F3E/E6F7C90B14AD49C4816B6EA4BE11752C] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS, with id=dag_1463533086249_0001_1
2016-05-18 00:58:06,502 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(260)) - Ignoring stop request for containerId: container_1463533086249_0001_00_000001
2016-05-18 00:58:06,503 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[764402181712400182BE4B6850478F3E/E6F7C90B14AD49C4816B6EA4BE11752C] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS, with id=dag_1463533086249_0001_1
2016-05-18 00:58:06,503 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...RESULTS.-1_VALUES_RESULTS, applicationId=application_1463533086249_0001
2016-05-18 00:58:06,507 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:06,507 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:06,507 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:06,507 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:06,507 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:06,508 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:06,508 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:06,508 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:06,509 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:06,509 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:06,509 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:06,509 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:06,510 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:06,510 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:06,515 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/764402181712400182BE4B6850478F3E/E6F7C90B14AD49C4816B6EA4BE11752C/.tez/application_1463533086249_0001
2016-05-18 00:58:06,516 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-18 00:58:06,516 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-18 00:58:06,516 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-18 00:58:06,517 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_RESULTS/_temporary
2016-05-18 00:58:06,517 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.290
2016-05-18 00:58:06,517 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_VALUES_RESULTS
2016-05-18 00:58:06,546 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:06,546 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:06,565 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:58:06,565 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:58:06,589 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-18 00:58:06,598 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-05-18 00:58:06,598 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:06,598 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:06,598 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:06,615 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:06,615 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_ALL
2016-05-18 00:58:06,626 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:06,626 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:06,626 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_ALL"]
2016-05-18 00:58:06,627 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:06,627 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:06,627 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:06,628 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-18 00:58:06,628 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:06,642 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:06,646 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:06,646 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:06,646 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:06,656 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D61D83E6ADF441538374DE4C270AB366/E4E32C0CCFC44F1F93423BB657B80EB7/.tez/application_1463533086646_0001 doesn't exist and is created
2016-05-18 00:58:06,672 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:06,672 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:06,673 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D61D83E6ADF441538374DE4C270AB366/E4E32C0CCFC44F1F93423BB657B80EB7/.tez/application_1463533086646_0001_wd
2016-05-18 00:58:06,683 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533086646_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:06,683 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:06,683 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:06,684 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:06,684 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:06,685 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:06,685 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:06,685 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:06,687 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:06,687 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:06,687 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:06,687 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:06,688 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:06,688 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:06,688 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:06,688 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:06,688 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:06,689 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:06,689 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:06,689 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:06,690 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:06,691 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:06,692 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:06,692 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D61D83E6ADF441538374DE4C270AB366/E4E32C0CCFC44F1F93423BB657B80EB7/.tez/application_1463533086646_0001_wd/localmode-log-dir
2016-05-18 00:58:06,692 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D61D83E6ADF441538374DE4C270AB366/E4E32C0CCFC44F1F93423BB657B80EB7/.tez/application_1463533086646_0001_wd/localmode-log-dir/history.txt.appattempt_1463533086646_0001_000000, maxErrors=10
2016-05-18 00:58:06,693 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D61D83E6ADF441538374DE4C270AB366/E4E32C0CCFC44F1F93423BB657B80EB7/.tez/application_1463533086646_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:06,693 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533086646_0001_000000, appSubmitTime=1463533086682, launchTime=1463533086682
2016-05-18 00:58:06,697 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:06,701 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:36957
2016-05-18 00:58:06,701 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533086646
2016-05-18 00:58:06,703 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533086646_0001_000000, startTime=1463533086703
2016-05-18 00:58:06,703 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:06,772 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:06,773 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:06,773 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.-1_VALUES_ALL, applicationId=application_1463533086646_0001, dagName=[D61D83E6ADF441538374DE4C270AB366/E4E32C0CCFC44F1F93423BB657B80EB7] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-18 00:58:06,792 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [D61D83E6ADF441538374DE4C270AB366/E4E32C0CCFC44F1F93423BB657B80EB7] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-18 00:58:06,792 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:06,793 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533086646_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D61D83E6ADF441538374DE4C270AB366/E4E32C0CCFC44F1F93423BB657B80EB7/.tez/application_1463533086646_0001_wd/localmode-log-dir/dag_1463533086646_0001_1.dot
2016-05-18 00:58:06,793 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D61D83E6ADF441538374DE4C270AB366/E4E32C0CCFC44F1F93423BB657B80EB7/.tez/application_1463533086646_0001_wd/localmode-log-dir/dag_1463533086646_0001_1-tez-dag.pb.txt
2016-05-18 00:58:06,797 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533086646_0001_1
2016-05-18 00:58:06,797 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:06,797 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [D61D83E6ADF441538374DE4C270AB366/E4E32C0CCFC44F1F93423BB657B80EB7] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-18 00:58:06 Running Dag: dag_1463533086646_0001_1
2016-05-18 00:58:06,806 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086646_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533086646_0001_1, submitTime=1463533086792
2016-05-18 00:58:06,807 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D61D83E6ADF441538374DE4C270AB366/E4E32C0CCFC44F1F93423BB657B80EB7/.tez/application_1463533086646_0001/tez-conf.pb" } size: 36913 timestamp: 1463533086000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:06,807 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D]
2016-05-18 00:58:06,807 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D]
2016-05-18 00:58:06,807 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:06,808 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:06,808 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086646_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533086646_0001_1, initTime=1463533086808
2016-05-18 00:58:06,808 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533086646_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:06,809 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:06,809 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086646_0001_1][Event:DAG_STARTED]: dagID=dag_1463533086646_0001_1, startTime=1463533086809
2016-05-18 00:58:06,809 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533086646_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:06,809 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: C730A9A15D73442E94F3A46BB5DCBD9D : {E812F3A5EE014098B765A96BEB02349F={InputName=E812F3A5EE014098B765A96BEB02349F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:06,810 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: E812F3A5EE014098B765A96BEB02349F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:06,810 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D]
2016-05-18 00:58:06,810 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D]
2016-05-18 00:58:06,810 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D]
2016-05-18 00:58:06,810 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D]
2016-05-18 00:58:06,811 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:06,811 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: E812F3A5EE014098B765A96BEB02349F on vertex vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D]
2016-05-18 00:58:06,812 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.-1_VALUES_ALL, applicationId=application_1463533086646_0001, dagName=[D61D83E6ADF441538374DE4C270AB366/E4E32C0CCFC44F1F93423BB657B80EB7] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-18 00:58:06,815 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input E812F3A5EE014098B765A96BEB02349F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:06,817 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:06,817 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:06,818 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:06,818 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:06,818 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: E812F3A5EE014098B765A96BEB02349F on vertex vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D]
2016-05-18 00:58:06,820 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:06,820 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {E812F3A5EE014098B765A96BEB02349F=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:06,821 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D] parallelism set to 1
2016-05-18 00:58:06,821 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:06,822 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D], numAdditionalOutputs=1
2016-05-18 00:58:06,822 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=7E441972A2CB4355B6D2F65F56A27993, vertex=vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:06,823 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for C730A9A15D73442E94F3A46BB5DCBD9D:7E441972A2CB4355B6D2F65F56A27993 using oldmapred API
2016-05-18 00:58:06,824 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=7E441972A2CB4355B6D2F65F56A27993, vertexName=C730A9A15D73442E94F3A46BB5DCBD9D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:06,825 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086646_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C730A9A15D73442E94F3A46BB5DCBD9D, vertexId=vertex_1463533086646_0001_1_00, initRequestedTime=1463533086809, initedTime=1463533086822, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:06,825 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D] with distanceFromRoot: 0
2016-05-18 00:58:06,825 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:06,825 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086646_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533086646_0001_1_00, startRequestedTime=1463533086811, startedTime=1463533086825
2016-05-18 00:58:06,826 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086646_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533086646_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:06,826 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:06,827 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in C730A9A15D73442E94F3A46BB5DCBD9D
2016-05-18 00:58:06,827 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D]
2016-05-18 00:58:06,828 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086646_0001_1][Event:TASK_STARTED]: vertexName=C730A9A15D73442E94F3A46BB5DCBD9D, taskId=task_1463533086646_0001_1_00_000000, scheduledTime=1463533086828, launchTime=1463533086828
2016-05-18 00:58:06,829 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D]
2016-05-18 00:58:06,829 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:06,831 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533086646_0001, with dag id: dag_1463533086646_0001_1
2016-05-18 00:58:06,831 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533086646_0001_00_000001, launchTime=1463533086831
2016-05-18 00:58:06,832 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533086646_0001_00_000001
2016-05-18 00:58:06,834 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533086646_0001_1_00_000000_0] started. Is using containerId: [container_1463533086646_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:06,834 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086646_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C730A9A15D73442E94F3A46BB5DCBD9D, taskAttemptId=attempt_1463533086646_0001_1_00_000000_0, startTime=1463533086833, containerId=container_1463533086646_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:06,835 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533086646_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533086646_0001_1_00_000000_0
2016-05-18 00:58:06 Starting to run new task attempt: attempt_1463533086646_0001_1_00_000000_0
2016-05-18 00:58:06,836 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533086646_0001_1_00_000000_0
2016-05-18 00:58:06,836 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:06,836 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:06,836 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D61D83E6ADF441538374DE4C270AB366/E4E32C0CCFC44F1F93423BB657B80EB7] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL, VertexName: C730A9A15D73442E94F3A46BB5DCBD9D, VertexParallelism: 1, TaskAttemptID:attempt_1463533086646_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E812F3A5EE014098B765A96BEB02349F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=7E441972A2CB4355B6D2F65F56A27993, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:06,836 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:06,837 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533086646_0001_1_00_000000_0
2016-05-18 00:58:06,837 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533086646_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:06,841 INFO input.MRInput (MRInput.java:initialize(440)) - E812F3A5EE014098B765A96BEB02349F using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:06,841 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: E812F3A5EE014098B765A96BEB02349F
2016-05-18 00:58:06,841 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533086646_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:06,842 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:06,849 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C730A9A15D73442E94F3A46BB5DCBD9D, ordinal: 0
2016-05-18 00:58:06,850 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C730A9A15D73442E94F3A46BB5DCBD9D, mem on start (mb), free: 146, total: 374, max: 672
2016-05-18 00:58:06,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:06,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:06,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:06,853 INFO output.MROutput (MROutput.java:initialize(421)) - 7E441972A2CB4355B6D2F65F56A27993: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:06,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:06,853 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[E812F3A5EE014098B765A96BEB02349F:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [7E441972A2CB4355B6D2F65F56A27993:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:06,853 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:06,853 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:06,853 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:06,854 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[E812F3A5EE014098B765A96BEB02349F:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [7E441972A2CB4355B6D2F65F56A27993:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:06,854 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:06,854 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: E812F3A5EE014098B765A96BEB02349F being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:06,854 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: E812F3A5EE014098B765A96BEB02349F
2016-05-18 00:58:06,854 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:06,854 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:06,854 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:06,855 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533086646_0001_1_00_000000_0
2016-05-18 00:58:06,856 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:06,856 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - E812F3A5EE014098B765A96BEB02349F initialized RecordReader from event
2016-05-18 00:58:06,858 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:06,859 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: E812F3A5EE014098B765A96BEB02349F
2016-05-18 00:58:06,859 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_ALL"], id: 7E441972A2CB4355B6D2F65F56A27993
2016-05-18 00:58:06,859 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_ALL"]
2016-05-18 00:58:06,860 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:06,860 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C730A9A15D73442E94F3A46BB5DCBD9D, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:06,861 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 00:58:06,866 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 00:58:06,870 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 00:58:06,875 INFO output.MROutput (MROutput.java:close(526)) - 7E441972A2CB4355B6D2F65F56A27993 closed
2016-05-18 00:58:06,875 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533086646_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:06,876 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533086646_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:06,877 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533086646_0001_1_00_000000_0: Counters: 8 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463533086861, Process_Duration=15, Process_End_Time=1463533086876, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-18 00:58:06,877 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(817)) - Joining on EventRouter
2016-05-18 00:58:06,877 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C730A9A15D73442E94F3A46BB5DCBD9D, mem on close (mb), free: 144, total: 374, max: 672
2016-05-18 00:58:06,877 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(851)) - Closed processor for vertex=C730A9A15D73442E94F3A46BB5DCBD9D, index=0, interruptedStatus=false
2016-05-18 00:58:06,878 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(887)) - Closed input for vertex=C730A9A15D73442E94F3A46BB5DCBD9D, sourceVertex=E812F3A5EE014098B765A96BEB02349F, interruptedStatus=true
2016-05-18 00:58:06,878 INFO output.MROutput (MROutput.java:close(526)) - 7E441972A2CB4355B6D2F65F56A27993 closed
2016-05-18 00:58:06,878 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(911)) - Closed input for vertex=C730A9A15D73442E94F3A46BB5DCBD9D, sourceVertex=7E441972A2CB4355B6D2F65F56A27993, interruptedStatus=true
2016-05-18 00:58:06,879 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533086646_0001_1_00_000000_0 : TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 00:58:06,879 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 00:58:06,879 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533086646_0001_00_000001
2016-05-18 00:58:06,879 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086646_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C730A9A15D73442E94F3A46BB5DCBD9D, taskAttemptId=attempt_1463533086646_0001_1_00_000000_0, creationTime=1463533086828, allocationTime=1463533086830, startTime=1463533086833, finishTime=1463533086879, timeTaken=46, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task: attempt_1463533086646_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
, counters=Counters: 9, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463533086861, Process_Duration=15, Process_End_Time=1463533086876, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-18 00:58:06,879 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533086646_0001_00_000001
2016-05-18 00:58:06,880 INFO impl.TaskImpl (TaskImpl.java:transition(1170)) - Failing task: task_1463533086646_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-18 00:58:06,880 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086646_0001_1][Event:TASK_FINISHED]: vertexName=C730A9A15D73442E94F3A46BB5DCBD9D, taskId=task_1463533086646_0001_1_00_000000, startTime=1463533086833, finishTime=1463533086880, timeTaken=47, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463533086646_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
], counters=Counters: 0
2016-05-18 00:58:06,881 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533086646_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, errorMessage='TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation'}
2016-05-18 00:58:06,881 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(294)) - Container: container_1463533086646_0001_00_000001 completed but with errors
2016-05-18 00:58:06,882 INFO impl.VertexImpl (VertexImpl.java:transition(3353)) - Failing vertex: vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D] because task failed: task_1463533086646_0001_1_00_000000
2016-05-18 00:58:06,882 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(2102)) - Killing tasks in vertex: vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D] due to trigger: OWN_TASK_FAILURE
2016-05-18 00:58:06,882 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D], tasks=1, failed=1, killed=0, success=0, completed=1, commits=0, err=OWN_TASK_FAILURE
2016-05-18 00:58:06,882 INFO impl.VertexImpl (VertexImpl.java:finishWithTerminationCause(2084)) - Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
2016-05-18 00:58:06,882 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3087)) - Invoking committer abort for vertex, vertexId=vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D]
2016-05-18 00:58:06,882 INFO impl.VertexImpl (VertexImpl.java:run(3095)) - Invoking committer abort for output=7E441972A2CB4355B6D2F65F56A27993, vertexId=vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D]
2016-05-18 00:58:06,888 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086646_0001_1][Event:VERTEX_FINISHED]: vertexName=C730A9A15D73442E94F3A46BB5DCBD9D, vertexId=vertex_1463533086646_0001_1_00, initRequestedTime=1463533086809, initedTime=1463533086822, startRequestedTime=1463533086811, startedTime=1463533086825, finishTime=1463533086884, timeTaken=59, status=FAILED, diagnostics=Task failed, taskId=task_1463533086646_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463533086646_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]]
Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
Vertex vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D] killed/failed due to:OWN_TASK_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=1463533086833, firstTasksToStart=[ task_1463533086646_0001_1_00_000000 ], lastTaskFinishTime=1463533086879, lastTasksToFinish=[ task_1463533086646_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-18 00:58:06,888 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-18 00:58:06,889 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086646_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533086646_0001_00_000001, stoppedTime=1463533086889, exitStatus=-1000
2016-05-18 00:58:06,889 INFO node.AMNodeImpl (AMNodeImpl.java:transition(298)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463533086646_0001_1_00_000000_0 failed: true container: container_1463533086646_0001_00_000001 numFailedTAs: 0
2016-05-18 00:58:06,889 INFO container.AMContainerImpl (AMContainerImpl.java:transition(993)) - Container container_1463533086646_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
2016-05-18 00:58:06,889 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-18 00:58:06,889 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=VERTEX_FAILURE
2016-05-18 00:58:06,890 INFO impl.DAGImpl (DAGImpl.java:finishWithTerminationCause(1350)) - DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
2016-05-18 00:58:06,890 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533086646_0001_1, queueSize=0
2016-05-18 00:58:06,891 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533086646_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533086646_0001_1, startTime=1463533086809, finishTime=1463533086890, timeTaken=81, status=FAILED, diagnostics=Vertex failed, vertexName=C730A9A15D73442E94F3A46BB5DCBD9D, vertexId=vertex_1463533086646_0001_1_00, diagnostics=[Task failed, taskId=task_1463533086646_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463533086646_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D] killed/failed due to:OWN_TASK_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0, counters=Counters: 4, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, AM_CPU_MILLISECONDS=140, AM_GC_TIME_MILLIS=0
2016-05-18 00:58:06,892 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3082)) - Ignoring multiple aborts for vertex: vertex_1463533086646_0001_1_00 [C730A9A15D73442E94F3A46BB5DCBD9D]
2016-05-18 00:58:06,892 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533086646_0001_1 finished with state: FAILED
2016-05-18 00:58:06,892 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533086646_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:06,892 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(260)) - Ignoring stop request for containerId: container_1463533086646_0001_00_000001
2016-05-18 00:58:06 Completed Dag: dag_1463533086646_0001_1
2016-05-18 00:58:06,893 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533086646_0001_1, dagState=FAILED
2016-05-18 00:58:06,893 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533086646_0001_1_post
2016-05-18 00:58:06,893 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:06,897 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 1
2016-05-18 00:58:06,897 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:06,897 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[D61D83E6ADF441538374DE4C270AB366/E4E32C0CCFC44F1F93423BB657B80EB7] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL, with id=dag_1463533086646_0001_1
2016-05-18 00:58:06,898 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[D61D83E6ADF441538374DE4C270AB366/E4E32C0CCFC44F1F93423BB657B80EB7] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL, with id=dag_1463533086646_0001_1
2016-05-18 00:58:06,903 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ld'_RESULTS.-1_VALUES_ALL, applicationId=application_1463533086646_0001
2016-05-18 00:58:06,905 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:06,905 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:06,906 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:06,906 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:06,906 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:06,906 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:06,907 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:06,907 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:06,907 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:06,907 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:06,908 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:06,908 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:06,908 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:06,909 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:06,912 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D61D83E6ADF441538374DE4C270AB366/E4E32C0CCFC44F1F93423BB657B80EB7/.tez/application_1463533086646_0001
2016-05-18 00:58:06,912 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-18 00:58:06,913 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-18 00:58:06,913 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-18 00:58:06,914 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_ALL/_temporary
2016-05-18 00:58:06,915 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.287
2016-05-18 00:58:06,915 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_VALUES_ALL
2016-05-18 00:58:06,940 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:06,940 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:06,992 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.006
2016-05-18 00:58:07,001 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.014
2016-05-18 00:58:07,001 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 00:58:07,001 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 00:58:07,001 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_ARGS_'field2'
2016-05-18 00:58:07,040 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:07,041 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:07,098 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-18 00:58:07,110 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-05-18 00:58:07,111 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:07,111 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:07,111 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:07,132 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:07,133 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_ARGS_RESULTS
2016-05-18 00:58:07,145 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:07,145 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:07,146 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_ARGS_RESULTS"]
2016-05-18 00:58:07,146 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:07,146 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:07,147 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:07,147 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...'_RESULTS.-1_ARGS_RESULTS
2016-05-18 00:58:07,148 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:07,169 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:07,175 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:07,175 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:07,175 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:07,187 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/49D1797C5FA34DC5AB65C993A712789B/D39794F058B34AEBA6BA4861C2415108/.tez/application_1463533087175_0001 doesn't exist and is created
2016-05-18 00:58:07,205 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:07,205 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:07,206 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/49D1797C5FA34DC5AB65C993A712789B/D39794F058B34AEBA6BA4861C2415108/.tez/application_1463533087175_0001_wd
2016-05-18 00:58:07,216 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533087175_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:07,217 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:07,217 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:07,218 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:07,218 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:07,219 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:07,219 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:07,220 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:07,221 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:07,221 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:07,222 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:07,222 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:07,222 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:07,222 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:07,222 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:07,223 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:07,223 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:07,223 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:07,224 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:07,224 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:07,225 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:07,226 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:07,226 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:07,227 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/49D1797C5FA34DC5AB65C993A712789B/D39794F058B34AEBA6BA4861C2415108/.tez/application_1463533087175_0001_wd/localmode-log-dir
2016-05-18 00:58:07,227 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/49D1797C5FA34DC5AB65C993A712789B/D39794F058B34AEBA6BA4861C2415108/.tez/application_1463533087175_0001_wd/localmode-log-dir/history.txt.appattempt_1463533087175_0001_000000, maxErrors=10
2016-05-18 00:58:07,228 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/49D1797C5FA34DC5AB65C993A712789B/D39794F058B34AEBA6BA4861C2415108/.tez/application_1463533087175_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:07,228 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533087175_0001_000000, appSubmitTime=1463533087216, launchTime=1463533087216
2016-05-18 00:58:07,230 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:07,231 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:58:07,232 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:58:07,234 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:37630
2016-05-18 00:58:07,234 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533087175
2016-05-18 00:58:07,237 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533087175_0001_000000, startTime=1463533087237
2016-05-18 00:58:07,237 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:07,306 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:07,306 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:07,306 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...'_RESULTS.-1_ARGS_RESULTS, applicationId=application_1463533087175_0001, dagName=[49D1797C5FA34DC5AB65C993A712789B/D39794F058B34AEBA6BA4861C2415108] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS
2016-05-18 00:58:07,326 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [49D1797C5FA34DC5AB65C993A712789B/D39794F058B34AEBA6BA4861C2415108] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS
2016-05-18 00:58:07,326 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:07,327 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533087175_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/49D1797C5FA34DC5AB65C993A712789B/D39794F058B34AEBA6BA4861C2415108/.tez/application_1463533087175_0001_wd/localmode-log-dir/dag_1463533087175_0001_1.dot
2016-05-18 00:58:07,327 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/49D1797C5FA34DC5AB65C993A712789B/D39794F058B34AEBA6BA4861C2415108/.tez/application_1463533087175_0001_wd/localmode-log-dir/dag_1463533087175_0001_1-tez-dag.pb.txt
2016-05-18 00:58:07,331 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533087175_0001_1
2016-05-18 00:58:07,331 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:07,331 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [49D1797C5FA34DC5AB65C993A712789B/D39794F058B34AEBA6BA4861C2415108] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS
2016-05-18 00:58:07 Running Dag: dag_1463533087175_0001_1
2016-05-18 00:58:07,342 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087175_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533087175_0001_1, submitTime=1463533087326
2016-05-18 00:58:07,343 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/49D1797C5FA34DC5AB65C993A712789B/D39794F058B34AEBA6BA4861C2415108/.tez/application_1463533087175_0001/tez-conf.pb" } size: 36913 timestamp: 1463533087000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:07,343 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2]
2016-05-18 00:58:07,343 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2]
2016-05-18 00:58:07,343 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:07,343 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:07,344 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087175_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533087175_0001_1, initTime=1463533087344
2016-05-18 00:58:07,344 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533087175_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:07,345 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:07,345 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087175_0001_1][Event:DAG_STARTED]: dagID=dag_1463533087175_0001_1, startTime=1463533087345
2016-05-18 00:58:07,345 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533087175_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:07,345 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: E7E95B4D79B840BB85AE3A2FDE50ABF2 : {1533408707E745F0B57451514E7458F2={InputName=1533408707E745F0B57451514E7458F2}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:07,345 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 1533408707E745F0B57451514E7458F2, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:07,345 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2]
2016-05-18 00:58:07,346 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2]
2016-05-18 00:58:07,346 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2]
2016-05-18 00:58:07,346 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2]
2016-05-18 00:58:07,346 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...'_RESULTS.-1_ARGS_RESULTS, applicationId=application_1463533087175_0001, dagName=[49D1797C5FA34DC5AB65C993A712789B/D39794F058B34AEBA6BA4861C2415108] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS
2016-05-18 00:58:07,346 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:07,347 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 1533408707E745F0B57451514E7458F2 on vertex vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2]
2016-05-18 00:58:07,349 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 1533408707E745F0B57451514E7458F2 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:07,351 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:07,351 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:07,351 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:07,352 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:07,352 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 1533408707E745F0B57451514E7458F2 on vertex vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2]
2016-05-18 00:58:07,353 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:07,354 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {1533408707E745F0B57451514E7458F2=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:07,355 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2] parallelism set to 1
2016-05-18 00:58:07,355 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:07,356 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2], numAdditionalOutputs=1
2016-05-18 00:58:07,356 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=1D6D63E647FD458298E729D0338813A6, vertex=vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:07,358 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for E7E95B4D79B840BB85AE3A2FDE50ABF2:1D6D63E647FD458298E729D0338813A6 using oldmapred API
2016-05-18 00:58:07,358 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=1D6D63E647FD458298E729D0338813A6, vertexName=E7E95B4D79B840BB85AE3A2FDE50ABF2, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:07,359 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087175_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E7E95B4D79B840BB85AE3A2FDE50ABF2, vertexId=vertex_1463533087175_0001_1_00, initRequestedTime=1463533087345, initedTime=1463533087356, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:07,360 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2] with distanceFromRoot: 0
2016-05-18 00:58:07,360 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:07,360 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087175_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533087175_0001_1_00, startRequestedTime=1463533087347, startedTime=1463533087360
2016-05-18 00:58:07,361 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087175_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533087175_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:07,361 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:07,361 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in E7E95B4D79B840BB85AE3A2FDE50ABF2
2016-05-18 00:58:07,362 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2]
2016-05-18 00:58:07,363 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087175_0001_1][Event:TASK_STARTED]: vertexName=E7E95B4D79B840BB85AE3A2FDE50ABF2, taskId=task_1463533087175_0001_1_00_000000, scheduledTime=1463533087362, launchTime=1463533087362
2016-05-18 00:58:07,363 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2]
2016-05-18 00:58:07,364 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:07,367 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533087175_0001, with dag id: dag_1463533087175_0001_1
2016-05-18 00:58:07,367 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533087175_0001_00_000001, launchTime=1463533087366
2016-05-18 00:58:07,368 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533087175_0001_00_000001
2016-05-18 00:58:07,370 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533087175_0001_1_00_000000_0] started. Is using containerId: [container_1463533087175_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:07,370 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087175_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E7E95B4D79B840BB85AE3A2FDE50ABF2, taskAttemptId=attempt_1463533087175_0001_1_00_000000_0, startTime=1463533087369, containerId=container_1463533087175_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:07,371 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533087175_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533087175_0001_1_00_000000_0
2016-05-18 00:58:07 Starting to run new task attempt: attempt_1463533087175_0001_1_00_000000_0
2016-05-18 00:58:07,372 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533087175_0001_1_00_000000_0
2016-05-18 00:58:07,372 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:07,372 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:07,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [49D1797C5FA34DC5AB65C993A712789B/D39794F058B34AEBA6BA4861C2415108] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS, VertexName: E7E95B4D79B840BB85AE3A2FDE50ABF2, VertexParallelism: 1, TaskAttemptID:attempt_1463533087175_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=1533408707E745F0B57451514E7458F2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=1D6D63E647FD458298E729D0338813A6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:07,372 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:07,373 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533087175_0001_1_00_000000_0
2016-05-18 00:58:07,373 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533087175_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:07,374 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533087175_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:07,378 INFO input.MRInput (MRInput.java:initialize(440)) - 1533408707E745F0B57451514E7458F2 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:07,379 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 1533408707E745F0B57451514E7458F2
2016-05-18 00:58:07,379 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:07,387 INFO output.MROutput (MROutput.java:initialize(421)) - 1D6D63E647FD458298E729D0338813A6: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:07,389 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E7E95B4D79B840BB85AE3A2FDE50ABF2, ordinal: 0
2016-05-18 00:58:07,389 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E7E95B4D79B840BB85AE3A2FDE50ABF2, mem on start (mb), free: 254, total: 377, max: 672
2016-05-18 00:58:07,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:07,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:07,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:07,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:07,390 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[1533408707E745F0B57451514E7458F2:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [1D6D63E647FD458298E729D0338813A6:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:07,390 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:07,390 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:07,390 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:07,390 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[1533408707E745F0B57451514E7458F2:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [1D6D63E647FD458298E729D0338813A6:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:07,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:07,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 1533408707E745F0B57451514E7458F2 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:07,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:07,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 1533408707E745F0B57451514E7458F2
2016-05-18 00:58:07,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:07,392 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:07,392 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533087175_0001_1_00_000000_0
2016-05-18 00:58:07,393 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:07,394 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 1533408707E745F0B57451514E7458F2 initialized RecordReader from event
2016-05-18 00:58:07,395 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:07,396 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 1533408707E745F0B57451514E7458F2
2016-05-18 00:58:07,396 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_ARGS_RESULTS"], id: 1D6D63E647FD458298E729D0338813A6
2016-05-18 00:58:07,396 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_ARGS_RESULTS"]
2016-05-18 00:58:07,397 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:07,397 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E7E95B4D79B840BB85AE3A2FDE50ABF2, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:07,400 INFO output.MROutput (MROutput.java:close(526)) - 1D6D63E647FD458298E729D0338813A6 closed
2016-05-18 00:58:07,400 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533087175_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:07,401 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533087175_0001_1_00_000000_0
2016-05-18 00:58:07,401 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E7E95B4D79B840BB85AE3A2FDE50ABF2, mem on close (mb), free: 252, total: 377, max: 672
2016-05-18 00:58:07,401 INFO output.MROutput (MROutput.java:close(526)) - 1D6D63E647FD458298E729D0338813A6 closed
2016-05-18 00:58:07,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:07 Completed running task attempt: attempt_1463533087175_0001_1_00_000000_0
2016-05-18 00:58:07,402 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533087175_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:07,402 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533087175_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:07,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533087175_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533087397, Process_Duration=4, Process_End_Time=1463533087401, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:07,403 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533087175_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:07,403 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:07,404 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:07,404 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:07,404 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533087175_0001_00_000001
2016-05-18 00:58:07,404 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087175_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E7E95B4D79B840BB85AE3A2FDE50ABF2, taskAttemptId=attempt_1463533087175_0001_1_00_000000_0, creationTime=1463533087363, allocationTime=1463533087366, startTime=1463533087369, finishTime=1463533087404, timeTaken=35, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:07,404 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533087175_0001_1_00_000000_0
2016-05-18 00:58:07,407 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087175_0001_1][Event:TASK_FINISHED]: vertexName=E7E95B4D79B840BB85AE3A2FDE50ABF2, taskId=task_1463533087175_0001_1_00_000000, startTime=1463533087369, finishTime=1463533087405, timeTaken=36, status=SUCCEEDED, successfulAttemptID=attempt_1463533087175_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533087397, Process_Duration=4, Process_End_Time=1463533087401, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:07,407 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:07,408 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2]
2016-05-18 00:58:07,409 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087175_0001_1][Event:VERTEX_FINISHED]: vertexName=E7E95B4D79B840BB85AE3A2FDE50ABF2, vertexId=vertex_1463533087175_0001_1_00, initRequestedTime=1463533087345, initedTime=1463533087356, startRequestedTime=1463533087347, startedTime=1463533087360, finishTime=1463533087408, timeTaken=48, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533087397, Process_Duration=4, Process_End_Time=1463533087401, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533087369, firstTasksToStart=[ task_1463533087175_0001_1_00_000000 ], lastTaskFinishTime=1463533087404, lastTasksToFinish=[ task_1463533087175_0001_1_00_000000 ], minTaskDuration=35, maxTaskDuration=35, avgTaskDuration=35.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533087175_0001_1_00_000000 ], longestDurationTasks=[ task_1463533087175_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:07,409 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:07,409 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087175_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533087175_0001_00_000001, stoppedTime=1463533087409, exitStatus=0
2016-05-18 00:58:07,410 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:07,410 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:07,410 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533087175_0001_1
2016-05-18 00:58:07,411 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087175_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533087175_0001_1
2016-05-18 00:58:07,411 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 1D6D63E647FD458298E729D0338813A6 for vertex: vertex_1463533087175_0001_1_00 [E7E95B4D79B840BB85AE3A2FDE50ABF2], outputName: 1D6D63E647FD458298E729D0338813A6
2016-05-18 00:58:07,411 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533087175_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:07,412 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533087175_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:07,414 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533087175_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:07,415 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533087175_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:07,415 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533087175_0001_00_000001, Exiting
2016-05-18 00:58:07,415 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533087175_0001_00_000001
2016-05-18 00:58:07,415 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533087175_0001_00_000001
2016-05-18 00:58:07,415 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533087175_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:07,415 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533087175_0001_00_000001 completed successfully
2016-05-18 00:58:07,419 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:1D6D63E647FD458298E729D0338813A6 of vertex/vertexGroup:E7E95B4D79B840BB85AE3A2FDE50ABF2 isVertexGroupOutput:false
2016-05-18 00:58:07,419 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:07,420 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533087175_0001_1, queueSize=0
2016-05-18 00:58:07,421 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087175_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533087175_0001_1, startTime=1463533087345, finishTime=1463533087419, timeTaken=74, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=150, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533087397, Process_Duration=4, Process_End_Time=1463533087401, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:07,421 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533087175_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:07,421 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533087175_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:07 Completed Dag: dag_1463533087175_0001_1
2016-05-18 00:58:07,421 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533087175_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:07,421 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533087175_0001_1_post
2016-05-18 00:58:07,422 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:07,422 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:07,422 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:07,422 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[49D1797C5FA34DC5AB65C993A712789B/D39794F058B34AEBA6BA4861C2415108] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS, with id=dag_1463533087175_0001_1
2016-05-18 00:58:07,422 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[49D1797C5FA34DC5AB65C993A712789B/D39794F058B34AEBA6BA4861C2415108] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS, with id=dag_1463533087175_0001_1
2016-05-18 00:58:07,429 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...'_RESULTS.-1_ARGS_RESULTS, applicationId=application_1463533087175_0001
2016-05-18 00:58:07,432 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:07,432 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:07,432 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:07,432 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:07,432 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:07,433 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:07,434 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:07,434 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:07,435 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:07,435 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:07,435 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:07,436 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:07,436 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:07,437 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:07,440 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/49D1797C5FA34DC5AB65C993A712789B/D39794F058B34AEBA6BA4861C2415108/.tez/application_1463533087175_0001
2016-05-18 00:58:07,441 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_ARGS_RESULTS/_temporary
2016-05-18 00:58:07,441 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.295
2016-05-18 00:58:07,513 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_ARGS_RESULTS
2016-05-18 00:58:07,539 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:07,539 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:07,573 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.006
2016-05-18 00:58:07,574 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.007
2016-05-18 00:58:07,574 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 00:58:07,575 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 00:58:07,575 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_ARGS_ALL
2016-05-18 00:58:07,600 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:07,600 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:07,639 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-18 00:58:07,646 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-18 00:58:07,646 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:07,647 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:07,647 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:07,663 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:07,664 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_-1
2016-05-18 00:58:07,674 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:07,675 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:07,675 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_-1"]
2016-05-18 00:58:07,675 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:07,675 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:07,675 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:07,676 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ld'_RESULTS.0_'field2'_-1
2016-05-18 00:58:07,676 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:07,694 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:07,700 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:07,700 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:07,700 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:07,712 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8E0C5133692942C6A00248047A0F9632/E365F5A150A34E4B935C80CA0F9FF7C3/.tez/application_1463533087700_0001 doesn't exist and is created
2016-05-18 00:58:07,733 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:07,733 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:07,735 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8E0C5133692942C6A00248047A0F9632/E365F5A150A34E4B935C80CA0F9FF7C3/.tez/application_1463533087700_0001_wd
2016-05-18 00:58:07,747 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533087700_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:07,748 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:07,748 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:07,749 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:07,749 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:07,749 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:07,749 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:07,750 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:07,752 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:07,752 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:07,752 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:07,753 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:07,753 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:07,753 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:07,754 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:07,754 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:07,755 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:07,755 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:07,756 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:07,756 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:07,758 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:07,760 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:07,760 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:07,761 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8E0C5133692942C6A00248047A0F9632/E365F5A150A34E4B935C80CA0F9FF7C3/.tez/application_1463533087700_0001_wd/localmode-log-dir
2016-05-18 00:58:07,761 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8E0C5133692942C6A00248047A0F9632/E365F5A150A34E4B935C80CA0F9FF7C3/.tez/application_1463533087700_0001_wd/localmode-log-dir/history.txt.appattempt_1463533087700_0001_000000, maxErrors=10
2016-05-18 00:58:07,762 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8E0C5133692942C6A00248047A0F9632/E365F5A150A34E4B935C80CA0F9FF7C3/.tez/application_1463533087700_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:07,762 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533087700_0001_000000, appSubmitTime=1463533087747, launchTime=1463533087747
2016-05-18 00:58:07,766 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:07,770 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:35858
2016-05-18 00:58:07,770 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533087700
2016-05-18 00:58:07,772 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533087700_0001_000000, startTime=1463533087772
2016-05-18 00:58:07,773 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:07,834 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:07,834 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:07,834 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.0_'field2'_-1, applicationId=application_1463533087700_0001, dagName=[8E0C5133692942C6A00248047A0F9632/E365F5A150A34E4B935C80CA0F9FF7C3] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1
2016-05-18 00:58:07,844 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:58:07,844 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:58:07,852 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [8E0C5133692942C6A00248047A0F9632/E365F5A150A34E4B935C80CA0F9FF7C3] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1
2016-05-18 00:58:07,853 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:07,854 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533087700_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8E0C5133692942C6A00248047A0F9632/E365F5A150A34E4B935C80CA0F9FF7C3/.tez/application_1463533087700_0001_wd/localmode-log-dir/dag_1463533087700_0001_1.dot
2016-05-18 00:58:07,855 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8E0C5133692942C6A00248047A0F9632/E365F5A150A34E4B935C80CA0F9FF7C3/.tez/application_1463533087700_0001_wd/localmode-log-dir/dag_1463533087700_0001_1-tez-dag.pb.txt
2016-05-18 00:58:07,860 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533087700_0001_1
2016-05-18 00:58:07,860 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:07,860 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [8E0C5133692942C6A00248047A0F9632/E365F5A150A34E4B935C80CA0F9FF7C3] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1
2016-05-18 00:58:07 Running Dag: dag_1463533087700_0001_1
2016-05-18 00:58:07,871 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087700_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533087700_0001_1, submitTime=1463533087853
2016-05-18 00:58:07,872 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8E0C5133692942C6A00248047A0F9632/E365F5A150A34E4B935C80CA0F9FF7C3/.tez/application_1463533087700_0001/tez-conf.pb" } size: 36913 timestamp: 1463533087000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:07,873 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6]
2016-05-18 00:58:07,873 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6]
2016-05-18 00:58:07,873 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:07,874 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:07,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087700_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533087700_0001_1, initTime=1463533087874
2016-05-18 00:58:07,875 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533087700_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:07,875 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:07,875 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087700_0001_1][Event:DAG_STARTED]: dagID=dag_1463533087700_0001_1, startTime=1463533087875
2016-05-18 00:58:07,876 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533087700_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:07,876 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: D1BB5DE6636948DFAAC0B2BBFE8F46A6 : {7256EE6E92644AEEA2CD1210F81DC10A={InputName=7256EE6E92644AEEA2CD1210F81DC10A}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:07,876 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 7256EE6E92644AEEA2CD1210F81DC10A, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:07,876 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6]
2016-05-18 00:58:07,876 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6]
2016-05-18 00:58:07,876 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6]
2016-05-18 00:58:07,877 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6]
2016-05-18 00:58:07,879 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:07,879 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.0_'field2'_-1, applicationId=application_1463533087700_0001, dagName=[8E0C5133692942C6A00248047A0F9632/E365F5A150A34E4B935C80CA0F9FF7C3] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1
2016-05-18 00:58:07,879 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 7256EE6E92644AEEA2CD1210F81DC10A on vertex vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6]
2016-05-18 00:58:07,882 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 7256EE6E92644AEEA2CD1210F81DC10A asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:07,883 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:07,884 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:07,884 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:07,884 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:07,884 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 7256EE6E92644AEEA2CD1210F81DC10A on vertex vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6]
2016-05-18 00:58:07,885 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:07,885 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {7256EE6E92644AEEA2CD1210F81DC10A=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:07,886 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6] parallelism set to 1
2016-05-18 00:58:07,887 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:07,887 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6], numAdditionalOutputs=1
2016-05-18 00:58:07,887 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=FF586159BA8640B2BDFFA8E57767CA2B, vertex=vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:07,890 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D1BB5DE6636948DFAAC0B2BBFE8F46A6:FF586159BA8640B2BDFFA8E57767CA2B using oldmapred API
2016-05-18 00:58:07,890 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=FF586159BA8640B2BDFFA8E57767CA2B, vertexName=D1BB5DE6636948DFAAC0B2BBFE8F46A6, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:07,891 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087700_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D1BB5DE6636948DFAAC0B2BBFE8F46A6, vertexId=vertex_1463533087700_0001_1_00, initRequestedTime=1463533087876, initedTime=1463533087887, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:07,891 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6] with distanceFromRoot: 0
2016-05-18 00:58:07,892 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:07,893 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087700_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533087700_0001_1_00, startRequestedTime=1463533087879, startedTime=1463533087892
2016-05-18 00:58:07,893 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087700_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533087700_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:07,894 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:07,894 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in D1BB5DE6636948DFAAC0B2BBFE8F46A6
2016-05-18 00:58:07,895 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6]
2016-05-18 00:58:07,896 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087700_0001_1][Event:TASK_STARTED]: vertexName=D1BB5DE6636948DFAAC0B2BBFE8F46A6, taskId=task_1463533087700_0001_1_00_000000, scheduledTime=1463533087895, launchTime=1463533087895
2016-05-18 00:58:07,896 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6]
2016-05-18 00:58:07,898 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:07,900 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533087700_0001_00_000001, launchTime=1463533087900
2016-05-18 00:58:07,901 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533087700_0001, with dag id: dag_1463533087700_0001_1
2016-05-18 00:58:07,907 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533087700_0001_00_000001
2016-05-18 00:58:07,908 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533087700_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533087700_0001_1_00_000000_0
2016-05-18 00:58:07,908 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533087700_0001_1_00_000000_0] started. Is using containerId: [container_1463533087700_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:07,908 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087700_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D1BB5DE6636948DFAAC0B2BBFE8F46A6, taskAttemptId=attempt_1463533087700_0001_1_00_000000_0, startTime=1463533087907, containerId=container_1463533087700_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:07 Starting to run new task attempt: attempt_1463533087700_0001_1_00_000000_0
2016-05-18 00:58:07,908 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533087700_0001_1_00_000000_0
2016-05-18 00:58:07,908 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:07,909 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:07,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8E0C5133692942C6A00248047A0F9632/E365F5A150A34E4B935C80CA0F9FF7C3] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1, VertexName: D1BB5DE6636948DFAAC0B2BBFE8F46A6, VertexParallelism: 1, TaskAttemptID:attempt_1463533087700_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7256EE6E92644AEEA2CD1210F81DC10A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FF586159BA8640B2BDFFA8E57767CA2B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:07,909 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:07,911 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533087700_0001_1_00_000000_0
2016-05-18 00:58:07,912 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533087700_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:07,913 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533087700_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:07,915 INFO input.MRInput (MRInput.java:initialize(440)) - 7256EE6E92644AEEA2CD1210F81DC10A using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:07,916 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 7256EE6E92644AEEA2CD1210F81DC10A
2016-05-18 00:58:07,918 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:07,924 INFO output.MROutput (MROutput.java:initialize(421)) - FF586159BA8640B2BDFFA8E57767CA2B: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:07,927 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D1BB5DE6636948DFAAC0B2BBFE8F46A6, ordinal: 0
2016-05-18 00:58:07,927 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D1BB5DE6636948DFAAC0B2BBFE8F46A6, mem on start (mb), free: 149, total: 377, max: 672
2016-05-18 00:58:07,927 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:07,928 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:07,928 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:07,928 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:07,928 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[7256EE6E92644AEEA2CD1210F81DC10A:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [FF586159BA8640B2BDFFA8E57767CA2B:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:07,928 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:07,928 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:07,929 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:07,929 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[7256EE6E92644AEEA2CD1210F81DC10A:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [FF586159BA8640B2BDFFA8E57767CA2B:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:07,929 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:07,929 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 7256EE6E92644AEEA2CD1210F81DC10A being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:07,929 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 7256EE6E92644AEEA2CD1210F81DC10A
2016-05-18 00:58:07,929 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:07,929 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:07,929 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:07,930 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533087700_0001_1_00_000000_0
2016-05-18 00:58:07,932 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:07,933 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 7256EE6E92644AEEA2CD1210F81DC10A initialized RecordReader from event
2016-05-18 00:58:07,933 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:07,934 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 7256EE6E92644AEEA2CD1210F81DC10A
2016-05-18 00:58:07,934 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_-1"], id: FF586159BA8640B2BDFFA8E57767CA2B
2016-05-18 00:58:07,934 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_-1"]
2016-05-18 00:58:07,934 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:07,935 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D1BB5DE6636948DFAAC0B2BBFE8F46A6, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:07,937 INFO output.MROutput (MROutput.java:close(526)) - FF586159BA8640B2BDFFA8E57767CA2B closed
2016-05-18 00:58:07,938 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533087700_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:07,939 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533087700_0001_1_00_000000_0
2016-05-18 00:58:07,939 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D1BB5DE6636948DFAAC0B2BBFE8F46A6, mem on close (mb), free: 147, total: 377, max: 672
2016-05-18 00:58:07,939 INFO output.MROutput (MROutput.java:close(526)) - FF586159BA8640B2BDFFA8E57767CA2B closed
2016-05-18 00:58:07,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:07 Completed running task attempt: attempt_1463533087700_0001_1_00_000000_0
2016-05-18 00:58:07,940 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533087700_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:07,940 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533087700_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:07,941 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533087700_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533087935, Process_Duration=4, Process_End_Time=1463533087939, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:07,942 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533087700_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:07,942 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:07,942 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087700_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D1BB5DE6636948DFAAC0B2BBFE8F46A6, taskAttemptId=attempt_1463533087700_0001_1_00_000000_0, creationTime=1463533087896, allocationTime=1463533087899, startTime=1463533087907, finishTime=1463533087942, timeTaken=35, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:07,942 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533087700_0001_1_00_000000_0
2016-05-18 00:58:07,942 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:07,943 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:07,943 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087700_0001_1][Event:TASK_FINISHED]: vertexName=D1BB5DE6636948DFAAC0B2BBFE8F46A6, taskId=task_1463533087700_0001_1_00_000000, startTime=1463533087907, finishTime=1463533087942, timeTaken=35, status=SUCCEEDED, successfulAttemptID=attempt_1463533087700_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533087935, Process_Duration=4, Process_End_Time=1463533087939, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:07,943 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533087700_0001_00_000001
2016-05-18 00:58:07,943 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:07,944 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6]
2016-05-18 00:58:07,945 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087700_0001_1][Event:VERTEX_FINISHED]: vertexName=D1BB5DE6636948DFAAC0B2BBFE8F46A6, vertexId=vertex_1463533087700_0001_1_00, initRequestedTime=1463533087876, initedTime=1463533087887, startRequestedTime=1463533087879, startedTime=1463533087892, finishTime=1463533087944, timeTaken=52, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533087935, Process_Duration=4, Process_End_Time=1463533087939, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533087907, firstTasksToStart=[ task_1463533087700_0001_1_00_000000 ], lastTaskFinishTime=1463533087942, lastTasksToFinish=[ task_1463533087700_0001_1_00_000000 ], minTaskDuration=35, maxTaskDuration=35, avgTaskDuration=35.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533087700_0001_1_00_000000 ], longestDurationTasks=[ task_1463533087700_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:07,945 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:07,945 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087700_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533087700_0001_00_000001, stoppedTime=1463533087945, exitStatus=0
2016-05-18 00:58:07,945 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:07,945 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:07,946 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533087700_0001_1
2016-05-18 00:58:07,946 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087700_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533087700_0001_1
2016-05-18 00:58:07,947 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: FF586159BA8640B2BDFFA8E57767CA2B for vertex: vertex_1463533087700_0001_1_00 [D1BB5DE6636948DFAAC0B2BBFE8F46A6], outputName: FF586159BA8640B2BDFFA8E57767CA2B
2016-05-18 00:58:07,949 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533087700_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:07,949 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533087700_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:07,953 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:FF586159BA8640B2BDFFA8E57767CA2B of vertex/vertexGroup:D1BB5DE6636948DFAAC0B2BBFE8F46A6 isVertexGroupOutput:false
2016-05-18 00:58:07,953 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:07,954 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533087700_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:07,954 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533087700_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:07,954 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533087700_0001_00_000001, Exiting
2016-05-18 00:58:07,955 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533087700_0001_00_000001
2016-05-18 00:58:07,955 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533087700_0001_00_000001
2016-05-18 00:58:07,955 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533087700_0001_1, queueSize=0
2016-05-18 00:58:07,955 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533087700_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533087700_0001_1, startTime=1463533087875, finishTime=1463533087953, timeTaken=78, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=180, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533087935, Process_Duration=4, Process_End_Time=1463533087939, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:07,956 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533087700_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:07,956 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533087700_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:07,956 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533087700_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:07 Completed Dag: dag_1463533087700_0001_1
2016-05-18 00:58:07,956 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533087700_0001_00_000001 completed successfully
2016-05-18 00:58:07,956 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533087700_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:07,958 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533087700_0001_1_post
2016-05-18 00:58:07,958 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:07,958 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 1
2016-05-18 00:58:07,958 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:07,959 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[8E0C5133692942C6A00248047A0F9632/E365F5A150A34E4B935C80CA0F9FF7C3] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1, with id=dag_1463533087700_0001_1
2016-05-18 00:58:07,959 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[8E0C5133692942C6A00248047A0F9632/E365F5A150A34E4B935C80CA0F9FF7C3] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1, with id=dag_1463533087700_0001_1
2016-05-18 00:58:07,967 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ld'_RESULTS.0_'field2'_-1, applicationId=application_1463533087700_0001
2016-05-18 00:58:07,969 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:07,969 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:07,969 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:07,969 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:07,969 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:07,970 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:07,970 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:07,972 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:07,972 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:07,972 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:07,972 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:07,972 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:07,973 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:07,974 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:07,976 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/8E0C5133692942C6A00248047A0F9632/E365F5A150A34E4B935C80CA0F9FF7C3/.tez/application_1463533087700_0001
2016-05-18 00:58:07,977 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_-1/_temporary
2016-05-18 00:58:07,977 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.302
2016-05-18 00:58:08,049 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_'field2'_-1
2016-05-18 00:58:08,088 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:08,088 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:08,135 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-18 00:58:08,156 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.036
2016-05-18 00:58:08,156 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:08,157 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:08,157 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:08,178 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:08,178 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_'field2'
2016-05-18 00:58:08,188 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:08,188 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:08,188 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_'field2'"]
2016-05-18 00:58:08,188 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:08,188 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:08,189 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:08,189 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...SULTS.0_'field2'_'field2'
2016-05-18 00:58:08,190 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:08,203 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:08,208 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:08,208 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:08,208 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:08,217 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/51B175FFF6694113AD6B8FC243E5038A/B9C62F36B0024BBCAF88B8854A2C41E6/.tez/application_1463533088208_0001 doesn't exist and is created
2016-05-18 00:58:08,240 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:08,241 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:08,242 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/51B175FFF6694113AD6B8FC243E5038A/B9C62F36B0024BBCAF88B8854A2C41E6/.tez/application_1463533088208_0001_wd
2016-05-18 00:58:08,254 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533088208_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:08,255 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:08,255 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:08,259 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:08,259 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:08,259 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:08,259 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:08,260 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:08,262 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:08,262 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:08,262 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:08,262 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:08,262 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:08,263 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:08,263 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:08,264 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:08,264 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:08,264 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:08,265 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:08,265 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:08,267 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:08,268 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:08,268 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:08,268 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/51B175FFF6694113AD6B8FC243E5038A/B9C62F36B0024BBCAF88B8854A2C41E6/.tez/application_1463533088208_0001_wd/localmode-log-dir
2016-05-18 00:58:08,269 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/51B175FFF6694113AD6B8FC243E5038A/B9C62F36B0024BBCAF88B8854A2C41E6/.tez/application_1463533088208_0001_wd/localmode-log-dir/history.txt.appattempt_1463533088208_0001_000000, maxErrors=10
2016-05-18 00:58:08,269 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/51B175FFF6694113AD6B8FC243E5038A/B9C62F36B0024BBCAF88B8854A2C41E6/.tez/application_1463533088208_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:08,271 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533088208_0001_000000, appSubmitTime=1463533088254, launchTime=1463533088254
2016-05-18 00:58:08,274 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:08,284 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:54173
2016-05-18 00:58:08,284 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533088208
2016-05-18 00:58:08,286 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533088208_0001_000000, startTime=1463533088286
2016-05-18 00:58:08,286 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:08,341 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:08,341 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:08,341 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...SULTS.0_'field2'_'field2', applicationId=application_1463533088208_0001, dagName=[51B175FFF6694113AD6B8FC243E5038A/B9C62F36B0024BBCAF88B8854A2C41E6] every.every/(1/1) ...SULTS.0_'field2'_'field2'
2016-05-18 00:58:08,358 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [51B175FFF6694113AD6B8FC243E5038A/B9C62F36B0024BBCAF88B8854A2C41E6] every.every/(1/1) ...SULTS.0_'field2'_'field2'
2016-05-18 00:58:08,358 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:08,359 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533088208_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/51B175FFF6694113AD6B8FC243E5038A/B9C62F36B0024BBCAF88B8854A2C41E6/.tez/application_1463533088208_0001_wd/localmode-log-dir/dag_1463533088208_0001_1.dot
2016-05-18 00:58:08,360 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/51B175FFF6694113AD6B8FC243E5038A/B9C62F36B0024BBCAF88B8854A2C41E6/.tez/application_1463533088208_0001_wd/localmode-log-dir/dag_1463533088208_0001_1-tez-dag.pb.txt
2016-05-18 00:58:08,363 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533088208_0001_1
2016-05-18 00:58:08,364 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:08,364 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [51B175FFF6694113AD6B8FC243E5038A/B9C62F36B0024BBCAF88B8854A2C41E6] every.every/(1/1) ...SULTS.0_'field2'_'field2'
2016-05-18 00:58:08 Running Dag: dag_1463533088208_0001_1
2016-05-18 00:58:08,373 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088208_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533088208_0001_1, submitTime=1463533088358
2016-05-18 00:58:08,374 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/51B175FFF6694113AD6B8FC243E5038A/B9C62F36B0024BBCAF88B8854A2C41E6/.tez/application_1463533088208_0001/tez-conf.pb" } size: 36913 timestamp: 1463533088000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:08,374 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7]
2016-05-18 00:58:08,375 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7]
2016-05-18 00:58:08,375 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:08,375 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:08,376 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088208_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533088208_0001_1, initTime=1463533088375
2016-05-18 00:58:08,376 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533088208_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:08,376 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:08,376 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088208_0001_1][Event:DAG_STARTED]: dagID=dag_1463533088208_0001_1, startTime=1463533088376
2016-05-18 00:58:08,377 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533088208_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:08,377 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: EA372DC89B7D42A8BAB7F156BFB94CF7 : {E0C9D9543CA04EDD96E174B97CB3199C={InputName=E0C9D9543CA04EDD96E174B97CB3199C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:08,377 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: E0C9D9543CA04EDD96E174B97CB3199C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:08,377 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7]
2016-05-18 00:58:08,378 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7]
2016-05-18 00:58:08,378 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...SULTS.0_'field2'_'field2', applicationId=application_1463533088208_0001, dagName=[51B175FFF6694113AD6B8FC243E5038A/B9C62F36B0024BBCAF88B8854A2C41E6] every.every/(1/1) ...SULTS.0_'field2'_'field2'
2016-05-18 00:58:08,378 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7]
2016-05-18 00:58:08,378 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7]
2016-05-18 00:58:08,379 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:08,379 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: E0C9D9543CA04EDD96E174B97CB3199C on vertex vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7]
2016-05-18 00:58:08,382 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input E0C9D9543CA04EDD96E174B97CB3199C asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:08,384 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:08,384 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:08,385 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:08,386 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:08,387 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: E0C9D9543CA04EDD96E174B97CB3199C on vertex vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7]
2016-05-18 00:58:08,392 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:08,392 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {E0C9D9543CA04EDD96E174B97CB3199C=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:08,395 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7] parallelism set to 1
2016-05-18 00:58:08,395 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:08,396 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7], numAdditionalOutputs=1
2016-05-18 00:58:08,396 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=C6896F09868D4159B72F2A9AE48C83B7, vertex=vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:08,398 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for EA372DC89B7D42A8BAB7F156BFB94CF7:C6896F09868D4159B72F2A9AE48C83B7 using oldmapred API
2016-05-18 00:58:08,398 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=C6896F09868D4159B72F2A9AE48C83B7, vertexName=EA372DC89B7D42A8BAB7F156BFB94CF7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:08,398 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533088208_0001, with dag id: dag_1463533088208_0001_1
2016-05-18 00:58:08,399 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088208_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EA372DC89B7D42A8BAB7F156BFB94CF7, vertexId=vertex_1463533088208_0001_1_00, initRequestedTime=1463533088377, initedTime=1463533088396, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:08,399 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7] with distanceFromRoot: 0
2016-05-18 00:58:08,399 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:08,400 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088208_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533088208_0001_1_00, startRequestedTime=1463533088379, startedTime=1463533088400
2016-05-18 00:58:08,401 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088208_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533088208_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:08,401 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:08,401 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in EA372DC89B7D42A8BAB7F156BFB94CF7
2016-05-18 00:58:08,401 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7]
2016-05-18 00:58:08,402 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088208_0001_1][Event:TASK_STARTED]: vertexName=EA372DC89B7D42A8BAB7F156BFB94CF7, taskId=task_1463533088208_0001_1_00_000000, scheduledTime=1463533088401, launchTime=1463533088401
2016-05-18 00:58:08,402 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:58:08,402 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:58:08,402 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7]
2016-05-18 00:58:08,403 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:08,405 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533088208_0001_00_000001, launchTime=1463533088405
2016-05-18 00:58:08,405 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533088208_0001_00_000001
2016-05-18 00:58:08,406 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533088208_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533088208_0001_1_00_000000_0
2016-05-18 00:58:08,406 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533088208_0001_1_00_000000_0] started. Is using containerId: [container_1463533088208_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:08,407 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088208_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EA372DC89B7D42A8BAB7F156BFB94CF7, taskAttemptId=attempt_1463533088208_0001_1_00_000000_0, startTime=1463533088406, containerId=container_1463533088208_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:08 Starting to run new task attempt: attempt_1463533088208_0001_1_00_000000_0
2016-05-18 00:58:08,407 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533088208_0001_1_00_000000_0
2016-05-18 00:58:08,407 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:08,407 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:08,407 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [51B175FFF6694113AD6B8FC243E5038A/B9C62F36B0024BBCAF88B8854A2C41E6] every.every/(1/1) ...SULTS.0_'field2'_'field2', VertexName: EA372DC89B7D42A8BAB7F156BFB94CF7, VertexParallelism: 1, TaskAttemptID:attempt_1463533088208_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E0C9D9543CA04EDD96E174B97CB3199C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C6896F09868D4159B72F2A9AE48C83B7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:08,408 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:08,408 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533088208_0001_1_00_000000_0
2016-05-18 00:58:08,409 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533088208_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:08,409 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533088208_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:08,411 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:08,419 INFO input.MRInput (MRInput.java:initialize(440)) - E0C9D9543CA04EDD96E174B97CB3199C using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:08,419 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: E0C9D9543CA04EDD96E174B97CB3199C
2016-05-18 00:58:08,420 INFO output.MROutput (MROutput.java:initialize(421)) - C6896F09868D4159B72F2A9AE48C83B7: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:08,424 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EA372DC89B7D42A8BAB7F156BFB94CF7, ordinal: 0
2016-05-18 00:58:08,425 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EA372DC89B7D42A8BAB7F156BFB94CF7, mem on start (mb), free: 255, total: 379, max: 672
2016-05-18 00:58:08,425 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:08,425 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:08,425 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:08,426 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:08,426 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[C6896F09868D4159B72F2A9AE48C83B7:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [E0C9D9543CA04EDD96E174B97CB3199C:INPUT:0:org.apache.tez.mapreduce.input.MRInput]
2016-05-18 00:58:08,426 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:08,427 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:08,427 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:08,428 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[C6896F09868D4159B72F2A9AE48C83B7:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [E0C9D9543CA04EDD96E174B97CB3199C:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0]
2016-05-18 00:58:08,428 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:08,428 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: E0C9D9543CA04EDD96E174B97CB3199C being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:08,428 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: E0C9D9543CA04EDD96E174B97CB3199C
2016-05-18 00:58:08,429 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:08,430 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:08,430 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:08,431 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533088208_0001_1_00_000000_0
2016-05-18 00:58:08,434 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:08,435 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: E0C9D9543CA04EDD96E174B97CB3199C
2016-05-18 00:58:08,435 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_'field2'"], id: C6896F09868D4159B72F2A9AE48C83B7
2016-05-18 00:58:08,436 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_'field2'"]
2016-05-18 00:58:08,436 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:08,438 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:08,438 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - E0C9D9543CA04EDD96E174B97CB3199C initialized RecordReader from event
2016-05-18 00:58:08,438 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EA372DC89B7D42A8BAB7F156BFB94CF7, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:08,442 INFO output.MROutput (MROutput.java:close(526)) - C6896F09868D4159B72F2A9AE48C83B7 closed
2016-05-18 00:58:08,442 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533088208_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:08,443 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533088208_0001_1_00_000000_0
2016-05-18 00:58:08,443 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EA372DC89B7D42A8BAB7F156BFB94CF7, mem on close (mb), free: 254, total: 379, max: 672
2016-05-18 00:58:08,444 INFO output.MROutput (MROutput.java:close(526)) - C6896F09868D4159B72F2A9AE48C83B7 closed
2016-05-18 00:58:08,444 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:08 Completed running task attempt: attempt_1463533088208_0001_1_00_000000_0
2016-05-18 00:58:08,445 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533088208_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:08,445 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533088208_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:08,446 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533088208_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533088439, Process_Duration=4, Process_End_Time=1463533088443, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:08,447 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533088208_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:08,448 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:08,448 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088208_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EA372DC89B7D42A8BAB7F156BFB94CF7, taskAttemptId=attempt_1463533088208_0001_1_00_000000_0, creationTime=1463533088402, allocationTime=1463533088404, startTime=1463533088406, finishTime=1463533088447, timeTaken=41, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:08,448 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:08,448 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:08,448 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533088208_0001_1_00_000000_0
2016-05-18 00:58:08,449 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088208_0001_1][Event:TASK_FINISHED]: vertexName=EA372DC89B7D42A8BAB7F156BFB94CF7, taskId=task_1463533088208_0001_1_00_000000, startTime=1463533088406, finishTime=1463533088449, timeTaken=43, status=SUCCEEDED, successfulAttemptID=attempt_1463533088208_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533088439, Process_Duration=4, Process_End_Time=1463533088443, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:08,449 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533088208_0001_00_000001
2016-05-18 00:58:08,449 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:08,450 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7]
2016-05-18 00:58:08,451 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088208_0001_1][Event:VERTEX_FINISHED]: vertexName=EA372DC89B7D42A8BAB7F156BFB94CF7, vertexId=vertex_1463533088208_0001_1_00, initRequestedTime=1463533088377, initedTime=1463533088396, startRequestedTime=1463533088379, startedTime=1463533088400, finishTime=1463533088450, timeTaken=50, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533088439, Process_Duration=4, Process_End_Time=1463533088443, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533088406, firstTasksToStart=[ task_1463533088208_0001_1_00_000000 ], lastTaskFinishTime=1463533088447, lastTasksToFinish=[ task_1463533088208_0001_1_00_000000 ], minTaskDuration=41, maxTaskDuration=41, avgTaskDuration=41.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533088208_0001_1_00_000000 ], longestDurationTasks=[ task_1463533088208_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:08,451 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:08,451 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088208_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533088208_0001_00_000001, stoppedTime=1463533088451, exitStatus=0
2016-05-18 00:58:08,452 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:08,452 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:08,452 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533088208_0001_1
2016-05-18 00:58:08,452 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088208_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533088208_0001_1
2016-05-18 00:58:08,452 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: C6896F09868D4159B72F2A9AE48C83B7 for vertex: vertex_1463533088208_0001_1_00 [EA372DC89B7D42A8BAB7F156BFB94CF7], outputName: C6896F09868D4159B72F2A9AE48C83B7
2016-05-18 00:58:08,454 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533088208_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:08,454 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533088208_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:08,459 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:C6896F09868D4159B72F2A9AE48C83B7 of vertex/vertexGroup:EA372DC89B7D42A8BAB7F156BFB94CF7 isVertexGroupOutput:false
2016-05-18 00:58:08,459 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:08,460 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533088208_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:08,460 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533088208_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:08,460 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533088208_0001_00_000001, Exiting
2016-05-18 00:58:08,460 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533088208_0001_00_000001
2016-05-18 00:58:08,461 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533088208_0001_00_000001
2016-05-18 00:58:08,461 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533088208_0001_1, queueSize=0
2016-05-18 00:58:08,462 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533088208_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:08,462 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533088208_0001_00_000001 completed successfully
2016-05-18 00:58:08,462 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088208_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533088208_0001_1, startTime=1463533088376, finishTime=1463533088460, timeTaken=84, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=170, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533088439, Process_Duration=4, Process_End_Time=1463533088443, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:08,462 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533088208_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:08,463 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533088208_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:08 Completed Dag: dag_1463533088208_0001_1
2016-05-18 00:58:08,463 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533088208_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:08,463 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533088208_0001_1_post
2016-05-18 00:58:08,463 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:08,463 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:08,463 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:08,464 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[51B175FFF6694113AD6B8FC243E5038A/B9C62F36B0024BBCAF88B8854A2C41E6] every.every/(1/1) ...SULTS.0_'field2'_'field2', with id=dag_1463533088208_0001_1
2016-05-18 00:58:08,464 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[51B175FFF6694113AD6B8FC243E5038A/B9C62F36B0024BBCAF88B8854A2C41E6] every.every/(1/1) ...SULTS.0_'field2'_'field2', with id=dag_1463533088208_0001_1
2016-05-18 00:58:08,470 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...SULTS.0_'field2'_'field2', applicationId=application_1463533088208_0001
2016-05-18 00:58:08,472 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:08,473 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:08,473 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:08,473 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:08,473 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:08,474 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:08,474 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:08,475 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:08,475 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:08,476 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:08,476 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:08,477 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:08,477 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:08,478 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:08,481 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/51B175FFF6694113AD6B8FC243E5038A/B9C62F36B0024BBCAF88B8854A2C41E6/.tez/application_1463533088208_0001
2016-05-18 00:58:08,482 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_'field2'/_temporary
2016-05-18 00:58:08,482 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.294
2016-05-18 00:58:08,554 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_'field2'_'field2'
2016-05-18 00:58:08,580 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:08,581 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:08,619 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-18 00:58:08,627 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-05-18 00:58:08,628 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:08,628 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:08,629 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:08,645 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:08,645 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_RESULTS
2016-05-18 00:58:08,655 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:08,656 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:08,656 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_RESULTS"]
2016-05-18 00:58:08,656 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:08,656 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:08,656 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:08,658 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ESULTS.0_'field2'_RESULTS
2016-05-18 00:58:08,658 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:08,672 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:08,676 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:08,677 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:08,677 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:08,686 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/81FD507948D04780A351CA90E1DBDD7B/E5BBB46F4CB1417D946EC98FF12CA5C4/.tez/application_1463533088677_0001 doesn't exist and is created
2016-05-18 00:58:08,704 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:08,704 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:08,705 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/81FD507948D04780A351CA90E1DBDD7B/E5BBB46F4CB1417D946EC98FF12CA5C4/.tez/application_1463533088677_0001_wd
2016-05-18 00:58:08,715 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533088677_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:08,716 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:08,717 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:08,719 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:08,719 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:08,719 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:08,719 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:08,721 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:08,722 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:08,722 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:08,723 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:08,723 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:08,723 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:08,723 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:08,723 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:08,723 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:08,724 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:08,724 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:08,724 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:08,725 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:08,726 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:08,727 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:08,727 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:08,728 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/81FD507948D04780A351CA90E1DBDD7B/E5BBB46F4CB1417D946EC98FF12CA5C4/.tez/application_1463533088677_0001_wd/localmode-log-dir
2016-05-18 00:58:08,728 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/81FD507948D04780A351CA90E1DBDD7B/E5BBB46F4CB1417D946EC98FF12CA5C4/.tez/application_1463533088677_0001_wd/localmode-log-dir/history.txt.appattempt_1463533088677_0001_000000, maxErrors=10
2016-05-18 00:58:08,729 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/81FD507948D04780A351CA90E1DBDD7B/E5BBB46F4CB1417D946EC98FF12CA5C4/.tez/application_1463533088677_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:08,729 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533088677_0001_000000, appSubmitTime=1463533088715, launchTime=1463533088715
2016-05-18 00:58:08,731 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:08,739 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:42328
2016-05-18 00:58:08,739 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533088677
2016-05-18 00:58:08,741 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533088677_0001_000000, startTime=1463533088741
2016-05-18 00:58:08,741 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:08,804 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:08,804 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:08,805 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ESULTS.0_'field2'_RESULTS, applicationId=application_1463533088677_0001, dagName=[81FD507948D04780A351CA90E1DBDD7B/E5BBB46F4CB1417D946EC98FF12CA5C4] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS
2016-05-18 00:58:08,827 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [81FD507948D04780A351CA90E1DBDD7B/E5BBB46F4CB1417D946EC98FF12CA5C4] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS
2016-05-18 00:58:08,828 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:08,829 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533088677_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/81FD507948D04780A351CA90E1DBDD7B/E5BBB46F4CB1417D946EC98FF12CA5C4/.tez/application_1463533088677_0001_wd/localmode-log-dir/dag_1463533088677_0001_1.dot
2016-05-18 00:58:08,829 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/81FD507948D04780A351CA90E1DBDD7B/E5BBB46F4CB1417D946EC98FF12CA5C4/.tez/application_1463533088677_0001_wd/localmode-log-dir/dag_1463533088677_0001_1-tez-dag.pb.txt
2016-05-18 00:58:08,843 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533088677_0001_1
2016-05-18 00:58:08,843 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:08,843 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [81FD507948D04780A351CA90E1DBDD7B/E5BBB46F4CB1417D946EC98FF12CA5C4] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS
2016-05-18 00:58:08 Running Dag: dag_1463533088677_0001_1
2016-05-18 00:58:08,854 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088677_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533088677_0001_1, submitTime=1463533088827
2016-05-18 00:58:08,860 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/81FD507948D04780A351CA90E1DBDD7B/E5BBB46F4CB1417D946EC98FF12CA5C4/.tez/application_1463533088677_0001/tez-conf.pb" } size: 36913 timestamp: 1463533088000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:08,860 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6]
2016-05-18 00:58:08,861 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6]
2016-05-18 00:58:08,861 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:08,861 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:08,863 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088677_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533088677_0001_1, initTime=1463533088863
2016-05-18 00:58:08,863 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533088677_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:08,864 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:08,864 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088677_0001_1][Event:DAG_STARTED]: dagID=dag_1463533088677_0001_1, startTime=1463533088864
2016-05-18 00:58:08,864 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533088677_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:08,864 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: F8883295E7C6436B812DCF2B2441D5D6 : {018927B3384043E5BAF0A4881B151C45={InputName=018927B3384043E5BAF0A4881B151C45}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:08,864 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 018927B3384043E5BAF0A4881B151C45, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:08,865 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6]
2016-05-18 00:58:08,865 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6]
2016-05-18 00:58:08,865 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6]
2016-05-18 00:58:08,866 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ESULTS.0_'field2'_RESULTS, applicationId=application_1463533088677_0001, dagName=[81FD507948D04780A351CA90E1DBDD7B/E5BBB46F4CB1417D946EC98FF12CA5C4] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS
2016-05-18 00:58:08,866 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6]
2016-05-18 00:58:08,867 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:08,867 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 018927B3384043E5BAF0A4881B151C45 on vertex vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6]
2016-05-18 00:58:08,871 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 018927B3384043E5BAF0A4881B151C45 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:08,872 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:08,873 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:08,873 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:08,873 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:08,873 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 018927B3384043E5BAF0A4881B151C45 on vertex vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6]
2016-05-18 00:58:08,875 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:08,875 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {018927B3384043E5BAF0A4881B151C45=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:08,875 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6] parallelism set to 1
2016-05-18 00:58:08,876 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:08,876 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6], numAdditionalOutputs=1
2016-05-18 00:58:08,876 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=C911ED1270644F378BDB831D0A7980E5, vertex=vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:08,879 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for F8883295E7C6436B812DCF2B2441D5D6:C911ED1270644F378BDB831D0A7980E5 using oldmapred API
2016-05-18 00:58:08,879 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=C911ED1270644F378BDB831D0A7980E5, vertexName=F8883295E7C6436B812DCF2B2441D5D6, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:08,880 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088677_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F8883295E7C6436B812DCF2B2441D5D6, vertexId=vertex_1463533088677_0001_1_00, initRequestedTime=1463533088864, initedTime=1463533088876, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:08,880 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6] with distanceFromRoot: 0
2016-05-18 00:58:08,881 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:08,881 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088677_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533088677_0001_1_00, startRequestedTime=1463533088867, startedTime=1463533088881
2016-05-18 00:58:08,881 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088677_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533088677_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:08,882 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:08,882 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in F8883295E7C6436B812DCF2B2441D5D6
2016-05-18 00:58:08,882 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6]
2016-05-18 00:58:08,883 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088677_0001_1][Event:TASK_STARTED]: vertexName=F8883295E7C6436B812DCF2B2441D5D6, taskId=task_1463533088677_0001_1_00_000000, scheduledTime=1463533088882, launchTime=1463533088882
2016-05-18 00:58:08,883 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6]
2016-05-18 00:58:08,884 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:08,886 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533088677_0001_00_000001, launchTime=1463533088886
2016-05-18 00:58:08,887 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533088677_0001_00_000001
2016-05-18 00:58:08,888 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533088677_0001_1_00_000000_0] started. Is using containerId: [container_1463533088677_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:08,888 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533088677_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533088677_0001_1_00_000000_0
2016-05-18 00:58:08,888 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088677_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F8883295E7C6436B812DCF2B2441D5D6, taskAttemptId=attempt_1463533088677_0001_1_00_000000_0, startTime=1463533088888, containerId=container_1463533088677_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:08 Starting to run new task attempt: attempt_1463533088677_0001_1_00_000000_0
2016-05-18 00:58:08,889 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533088677_0001_1_00_000000_0
2016-05-18 00:58:08,889 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:08,889 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:08,889 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533088677_0001, with dag id: dag_1463533088677_0001_1
2016-05-18 00:58:08,891 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [81FD507948D04780A351CA90E1DBDD7B/E5BBB46F4CB1417D946EC98FF12CA5C4] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS, VertexName: F8883295E7C6436B812DCF2B2441D5D6, VertexParallelism: 1, TaskAttemptID:attempt_1463533088677_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=018927B3384043E5BAF0A4881B151C45, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C911ED1270644F378BDB831D0A7980E5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:08,891 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:08,892 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533088677_0001_1_00_000000_0
2016-05-18 00:58:08,896 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:08,896 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533088677_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:08,902 INFO input.MRInput (MRInput.java:initialize(440)) - 018927B3384043E5BAF0A4881B151C45 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:08,902 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533088677_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:08,902 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 018927B3384043E5BAF0A4881B151C45
2016-05-18 00:58:08,911 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F8883295E7C6436B812DCF2B2441D5D6, ordinal: 0
2016-05-18 00:58:08,911 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F8883295E7C6436B812DCF2B2441D5D6, mem on start (mb), free: 167, total: 379, max: 672
2016-05-18 00:58:08,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:08,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:08,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:08,912 INFO output.MROutput (MROutput.java:initialize(421)) - C911ED1270644F378BDB831D0A7980E5: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:08,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:08,912 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[018927B3384043E5BAF0A4881B151C45:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [C911ED1270644F378BDB831D0A7980E5:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:08,913 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:08,913 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:08,913 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:08,914 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[018927B3384043E5BAF0A4881B151C45:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [C911ED1270644F378BDB831D0A7980E5:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:08,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:08,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 018927B3384043E5BAF0A4881B151C45 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:08,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 018927B3384043E5BAF0A4881B151C45
2016-05-18 00:58:08,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:08,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:08,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:08,915 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533088677_0001_1_00_000000_0
2016-05-18 00:58:08,917 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:08,917 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 018927B3384043E5BAF0A4881B151C45 initialized RecordReader from event
2016-05-18 00:58:08,920 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:08,921 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 018927B3384043E5BAF0A4881B151C45
2016-05-18 00:58:08,921 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_RESULTS"], id: C911ED1270644F378BDB831D0A7980E5
2016-05-18 00:58:08,921 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_RESULTS"]
2016-05-18 00:58:08,921 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:08,921 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F8883295E7C6436B812DCF2B2441D5D6, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:08,924 INFO output.MROutput (MROutput.java:close(526)) - C911ED1270644F378BDB831D0A7980E5 closed
2016-05-18 00:58:08,925 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533088677_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:08,925 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533088677_0001_1_00_000000_0
2016-05-18 00:58:08,926 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F8883295E7C6436B812DCF2B2441D5D6, mem on close (mb), free: 165, total: 379, max: 672
2016-05-18 00:58:08,926 INFO output.MROutput (MROutput.java:close(526)) - C911ED1270644F378BDB831D0A7980E5 closed
2016-05-18 00:58:08,926 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:08 Completed running task attempt: attempt_1463533088677_0001_1_00_000000_0
2016-05-18 00:58:08,926 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533088677_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:08,927 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533088677_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:08,927 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533088677_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533088921, Process_Duration=4, Process_End_Time=1463533088925, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:08,928 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533088677_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:08,928 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:08,929 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:08,929 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:08,929 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088677_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F8883295E7C6436B812DCF2B2441D5D6, taskAttemptId=attempt_1463533088677_0001_1_00_000000_0, creationTime=1463533088883, allocationTime=1463533088886, startTime=1463533088888, finishTime=1463533088929, timeTaken=41, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:08,929 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533088677_0001_1_00_000000_0
2016-05-18 00:58:08,930 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088677_0001_1][Event:TASK_FINISHED]: vertexName=F8883295E7C6436B812DCF2B2441D5D6, taskId=task_1463533088677_0001_1_00_000000, startTime=1463533088888, finishTime=1463533088930, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1463533088677_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533088921, Process_Duration=4, Process_End_Time=1463533088925, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:08,931 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:08,931 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6]
2016-05-18 00:58:08,932 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533088677_0001_00_000001
2016-05-18 00:58:08,932 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088677_0001_1][Event:VERTEX_FINISHED]: vertexName=F8883295E7C6436B812DCF2B2441D5D6, vertexId=vertex_1463533088677_0001_1_00, initRequestedTime=1463533088864, initedTime=1463533088876, startRequestedTime=1463533088867, startedTime=1463533088881, finishTime=1463533088931, timeTaken=50, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533088921, Process_Duration=4, Process_End_Time=1463533088925, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533088888, firstTasksToStart=[ task_1463533088677_0001_1_00_000000 ], lastTaskFinishTime=1463533088929, lastTasksToFinish=[ task_1463533088677_0001_1_00_000000 ], minTaskDuration=41, maxTaskDuration=41, avgTaskDuration=41.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533088677_0001_1_00_000000 ], longestDurationTasks=[ task_1463533088677_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:08,932 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:08,932 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088677_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533088677_0001_00_000001, stoppedTime=1463533088932, exitStatus=0
2016-05-18 00:58:08,933 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:08,933 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:08,933 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533088677_0001_1
2016-05-18 00:58:08,933 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088677_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533088677_0001_1
2016-05-18 00:58:08,933 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: C911ED1270644F378BDB831D0A7980E5 for vertex: vertex_1463533088677_0001_1_00 [F8883295E7C6436B812DCF2B2441D5D6], outputName: C911ED1270644F378BDB831D0A7980E5
2016-05-18 00:58:08,933 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533088677_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:08,934 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533088677_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:08,940 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:C911ED1270644F378BDB831D0A7980E5 of vertex/vertexGroup:F8883295E7C6436B812DCF2B2441D5D6 isVertexGroupOutput:false
2016-05-18 00:58:08,940 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:08,941 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533088677_0001_1, queueSize=0
2016-05-18 00:58:08,942 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533088677_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533088677_0001_1, startTime=1463533088864, finishTime=1463533088940, timeTaken=76, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=160, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533088921, Process_Duration=4, Process_End_Time=1463533088925, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:08,942 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533088677_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:08,942 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533088677_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:08,942 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533088677_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:08,942 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533088677_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:08,942 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533088677_0001_00_000001, Exiting
2016-05-18 00:58:08,942 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533088677_0001_00_000001
2016-05-18 00:58:08 Completed Dag: dag_1463533088677_0001_1
2016-05-18 00:58:08,943 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533088677_0001_00_000001
2016-05-18 00:58:08,943 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533088677_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:08,943 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533088677_0001_1_post
2016-05-18 00:58:08,943 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:08,943 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:08,943 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:08,943 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[81FD507948D04780A351CA90E1DBDD7B/E5BBB46F4CB1417D946EC98FF12CA5C4] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS, with id=dag_1463533088677_0001_1
2016-05-18 00:58:08,944 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[81FD507948D04780A351CA90E1DBDD7B/E5BBB46F4CB1417D946EC98FF12CA5C4] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS, with id=dag_1463533088677_0001_1
2016-05-18 00:58:08,944 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533088677_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:08,944 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533088677_0001_00_000001 completed successfully
2016-05-18 00:58:08,948 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ESULTS.0_'field2'_RESULTS, applicationId=application_1463533088677_0001
2016-05-18 00:58:08,951 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:08,951 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:08,951 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:08,951 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:08,951 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:08,952 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:08,952 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:08,952 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:08,953 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:08,953 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:08,953 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:08,953 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:08,954 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:08,954 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:08,958 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/81FD507948D04780A351CA90E1DBDD7B/E5BBB46F4CB1417D946EC98FF12CA5C4/.tez/application_1463533088677_0001
2016-05-18 00:58:08,958 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_RESULTS/_temporary
2016-05-18 00:58:08,958 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.303
2016-05-18 00:58:08,962 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:58:08,962 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:58:09,032 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_'field2'_RESULTS
2016-05-18 00:58:09,055 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:09,055 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:09,095 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-18 00:58:09,106 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-05-18 00:58:09,106 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:09,107 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:09,107 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:09,129 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:09,131 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_ALL
2016-05-18 00:58:09,142 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:09,143 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:09,143 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_ALL"]
2016-05-18 00:58:09,143 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:09,144 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:09,145 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:09,146 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...d'_RESULTS.0_'field2'_ALL
2016-05-18 00:58:09,146 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:09,174 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:09,179 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:09,179 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:09,180 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:09,189 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/190F25D924E04B14AD0AAEA7374A8CC8/B96B7FC7A7EC45CA9306149E5B34F5B1/.tez/application_1463533089179_0001 doesn't exist and is created
2016-05-18 00:58:09,203 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:09,203 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:09,204 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/190F25D924E04B14AD0AAEA7374A8CC8/B96B7FC7A7EC45CA9306149E5B34F5B1/.tez/application_1463533089179_0001_wd
2016-05-18 00:58:09,213 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533089179_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:09,213 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:09,214 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:09,215 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:09,215 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:09,215 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:09,215 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:09,216 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:09,217 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:09,217 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:09,218 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:09,218 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:09,218 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:09,218 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:09,218 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:09,219 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:09,219 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:09,219 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:09,219 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:09,220 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:09,221 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:09,222 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:09,222 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:09,222 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/190F25D924E04B14AD0AAEA7374A8CC8/B96B7FC7A7EC45CA9306149E5B34F5B1/.tez/application_1463533089179_0001_wd/localmode-log-dir
2016-05-18 00:58:09,222 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/190F25D924E04B14AD0AAEA7374A8CC8/B96B7FC7A7EC45CA9306149E5B34F5B1/.tez/application_1463533089179_0001_wd/localmode-log-dir/history.txt.appattempt_1463533089179_0001_000000, maxErrors=10
2016-05-18 00:58:09,223 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/190F25D924E04B14AD0AAEA7374A8CC8/B96B7FC7A7EC45CA9306149E5B34F5B1/.tez/application_1463533089179_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:09,223 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533089179_0001_000000, appSubmitTime=1463533089213, launchTime=1463533089213
2016-05-18 00:58:09,227 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:09,230 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:50937
2016-05-18 00:58:09,231 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533089179
2016-05-18 00:58:09,232 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533089179_0001_000000, startTime=1463533089232
2016-05-18 00:58:09,233 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:09,303 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:09,303 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:09,304 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...d'_RESULTS.0_'field2'_ALL, applicationId=application_1463533089179_0001, dagName=[190F25D924E04B14AD0AAEA7374A8CC8/B96B7FC7A7EC45CA9306149E5B34F5B1] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL
2016-05-18 00:58:09,320 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [190F25D924E04B14AD0AAEA7374A8CC8/B96B7FC7A7EC45CA9306149E5B34F5B1] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL
2016-05-18 00:58:09,321 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:09,322 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533089179_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/190F25D924E04B14AD0AAEA7374A8CC8/B96B7FC7A7EC45CA9306149E5B34F5B1/.tez/application_1463533089179_0001_wd/localmode-log-dir/dag_1463533089179_0001_1.dot
2016-05-18 00:58:09,322 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/190F25D924E04B14AD0AAEA7374A8CC8/B96B7FC7A7EC45CA9306149E5B34F5B1/.tez/application_1463533089179_0001_wd/localmode-log-dir/dag_1463533089179_0001_1-tez-dag.pb.txt
2016-05-18 00:58:09,326 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533089179_0001_1
2016-05-18 00:58:09,326 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:09,327 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [190F25D924E04B14AD0AAEA7374A8CC8/B96B7FC7A7EC45CA9306149E5B34F5B1] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL
2016-05-18 00:58:09 Running Dag: dag_1463533089179_0001_1
2016-05-18 00:58:09,336 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089179_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533089179_0001_1, submitTime=1463533089321
2016-05-18 00:58:09,337 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/190F25D924E04B14AD0AAEA7374A8CC8/B96B7FC7A7EC45CA9306149E5B34F5B1/.tez/application_1463533089179_0001/tez-conf.pb" } size: 36913 timestamp: 1463533089000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:09,337 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F]
2016-05-18 00:58:09,338 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F]
2016-05-18 00:58:09,338 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:09,338 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:09,338 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089179_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533089179_0001_1, initTime=1463533089338
2016-05-18 00:58:09,339 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533089179_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:09,339 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:09,339 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089179_0001_1][Event:DAG_STARTED]: dagID=dag_1463533089179_0001_1, startTime=1463533089339
2016-05-18 00:58:09,339 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533089179_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:09,339 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: C8B05AF59A604D8682A94C9622DD9A7F : {1413991E1B114DBBA0814583CD101C19={InputName=1413991E1B114DBBA0814583CD101C19}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:09,339 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 1413991E1B114DBBA0814583CD101C19, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:09,340 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F]
2016-05-18 00:58:09,340 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F]
2016-05-18 00:58:09,340 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F]
2016-05-18 00:58:09,340 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F]
2016-05-18 00:58:09,340 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:09,342 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 1413991E1B114DBBA0814583CD101C19 on vertex vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F]
2016-05-18 00:58:09,344 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...d'_RESULTS.0_'field2'_ALL, applicationId=application_1463533089179_0001, dagName=[190F25D924E04B14AD0AAEA7374A8CC8/B96B7FC7A7EC45CA9306149E5B34F5B1] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL
2016-05-18 00:58:09,345 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 1413991E1B114DBBA0814583CD101C19 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:09,346 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:09,346 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:09,346 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:09,346 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:09,347 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 1413991E1B114DBBA0814583CD101C19 on vertex vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F]
2016-05-18 00:58:09,348 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:09,348 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {1413991E1B114DBBA0814583CD101C19=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:09,349 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F] parallelism set to 1
2016-05-18 00:58:09,349 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:09,350 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F], numAdditionalOutputs=1
2016-05-18 00:58:09,350 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=6DB8BA4CDB894DC082CB0C8E74AB7A8E, vertex=vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:09,352 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for C8B05AF59A604D8682A94C9622DD9A7F:6DB8BA4CDB894DC082CB0C8E74AB7A8E using oldmapred API
2016-05-18 00:58:09,353 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=6DB8BA4CDB894DC082CB0C8E74AB7A8E, vertexName=C8B05AF59A604D8682A94C9622DD9A7F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:09,353 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089179_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C8B05AF59A604D8682A94C9622DD9A7F, vertexId=vertex_1463533089179_0001_1_00, initRequestedTime=1463533089339, initedTime=1463533089350, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:09,354 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F] with distanceFromRoot: 0
2016-05-18 00:58:09,354 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:09,354 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089179_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533089179_0001_1_00, startRequestedTime=1463533089341, startedTime=1463533089354
2016-05-18 00:58:09,354 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089179_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533089179_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:09,354 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:09,355 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in C8B05AF59A604D8682A94C9622DD9A7F
2016-05-18 00:58:09,355 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F]
2016-05-18 00:58:09,355 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089179_0001_1][Event:TASK_STARTED]: vertexName=C8B05AF59A604D8682A94C9622DD9A7F, taskId=task_1463533089179_0001_1_00_000000, scheduledTime=1463533089355, launchTime=1463533089355
2016-05-18 00:58:09,355 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F]
2016-05-18 00:58:09,356 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:09,358 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533089179_0001_00_000001, launchTime=1463533089358
2016-05-18 00:58:09,359 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533089179_0001_00_000001
2016-05-18 00:58:09,361 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533089179_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533089179_0001_1_00_000000_0
2016-05-18 00:58:09,361 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533089179_0001_1_00_000000_0] started. Is using containerId: [container_1463533089179_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:09,361 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089179_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C8B05AF59A604D8682A94C9622DD9A7F, taskAttemptId=attempt_1463533089179_0001_1_00_000000_0, startTime=1463533089361, containerId=container_1463533089179_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:09 Starting to run new task attempt: attempt_1463533089179_0001_1_00_000000_0
2016-05-18 00:58:09,361 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533089179_0001_1_00_000000_0
2016-05-18 00:58:09,361 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:09,361 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:09,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [190F25D924E04B14AD0AAEA7374A8CC8/B96B7FC7A7EC45CA9306149E5B34F5B1] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL, VertexName: C8B05AF59A604D8682A94C9622DD9A7F, VertexParallelism: 1, TaskAttemptID:attempt_1463533089179_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=1413991E1B114DBBA0814583CD101C19, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=6DB8BA4CDB894DC082CB0C8E74AB7A8E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:09,362 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:09,363 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533089179_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:09,364 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533089179_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:09,364 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533089179_0001_1_00_000000_0
2016-05-18 00:58:09,371 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:09,371 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533089179_0001, with dag id: dag_1463533089179_0001_1
2016-05-18 00:58:09,373 INFO input.MRInput (MRInput.java:initialize(440)) - 1413991E1B114DBBA0814583CD101C19 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:09,373 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 1413991E1B114DBBA0814583CD101C19
2016-05-18 00:58:09,380 INFO output.MROutput (MROutput.java:initialize(421)) - 6DB8BA4CDB894DC082CB0C8E74AB7A8E: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:09,381 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C8B05AF59A604D8682A94C9622DD9A7F, ordinal: 0
2016-05-18 00:58:09,381 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C8B05AF59A604D8682A94C9622DD9A7F, mem on start (mb), free: 283, total: 381, max: 672
2016-05-18 00:58:09,381 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:09,381 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:09,382 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:09,382 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:09,382 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[1413991E1B114DBBA0814583CD101C19:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [6DB8BA4CDB894DC082CB0C8E74AB7A8E:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:09,383 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:09,383 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:09,383 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:09,383 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[1413991E1B114DBBA0814583CD101C19:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [6DB8BA4CDB894DC082CB0C8E74AB7A8E:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:09,384 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:09,384 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 1413991E1B114DBBA0814583CD101C19 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:09,384 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 1413991E1B114DBBA0814583CD101C19
2016-05-18 00:58:09,384 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:09,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:09,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:09,385 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533089179_0001_1_00_000000_0
2016-05-18 00:58:09,387 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:09,387 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 1413991E1B114DBBA0814583CD101C19 initialized RecordReader from event
2016-05-18 00:58:09,388 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:09,389 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 1413991E1B114DBBA0814583CD101C19
2016-05-18 00:58:09,389 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_ALL"], id: 6DB8BA4CDB894DC082CB0C8E74AB7A8E
2016-05-18 00:58:09,390 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_ALL"]
2016-05-18 00:58:09,390 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:09,390 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C8B05AF59A604D8682A94C9622DD9A7F, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:09,393 INFO output.MROutput (MROutput.java:close(526)) - 6DB8BA4CDB894DC082CB0C8E74AB7A8E closed
2016-05-18 00:58:09,393 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533089179_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:09,394 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533089179_0001_1_00_000000_0
2016-05-18 00:58:09,394 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C8B05AF59A604D8682A94C9622DD9A7F, mem on close (mb), free: 281, total: 381, max: 672
2016-05-18 00:58:09,394 INFO output.MROutput (MROutput.java:close(526)) - 6DB8BA4CDB894DC082CB0C8E74AB7A8E closed
2016-05-18 00:58:09,394 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:09 Completed running task attempt: attempt_1463533089179_0001_1_00_000000_0
2016-05-18 00:58:09,394 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533089179_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:09,395 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533089179_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:09,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533089179_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533089390, Process_Duration=4, Process_End_Time=1463533089394, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:09,396 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533089179_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:09,396 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:09,396 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:09,396 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:09,397 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533089179_0001_00_000001
2016-05-18 00:58:09,397 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089179_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C8B05AF59A604D8682A94C9622DD9A7F, taskAttemptId=attempt_1463533089179_0001_1_00_000000_0, creationTime=1463533089355, allocationTime=1463533089357, startTime=1463533089361, finishTime=1463533089397, timeTaken=36, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:09,397 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533089179_0001_1_00_000000_0
2016-05-18 00:58:09,398 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089179_0001_1][Event:TASK_FINISHED]: vertexName=C8B05AF59A604D8682A94C9622DD9A7F, taskId=task_1463533089179_0001_1_00_000000, startTime=1463533089361, finishTime=1463533089398, timeTaken=37, status=SUCCEEDED, successfulAttemptID=attempt_1463533089179_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533089390, Process_Duration=4, Process_End_Time=1463533089394, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:09,398 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:09,398 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F]
2016-05-18 00:58:09,400 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089179_0001_1][Event:VERTEX_FINISHED]: vertexName=C8B05AF59A604D8682A94C9622DD9A7F, vertexId=vertex_1463533089179_0001_1_00, initRequestedTime=1463533089339, initedTime=1463533089350, startRequestedTime=1463533089341, startedTime=1463533089354, finishTime=1463533089398, timeTaken=44, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533089390, Process_Duration=4, Process_End_Time=1463533089394, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533089361, firstTasksToStart=[ task_1463533089179_0001_1_00_000000 ], lastTaskFinishTime=1463533089397, lastTasksToFinish=[ task_1463533089179_0001_1_00_000000 ], minTaskDuration=36, maxTaskDuration=36, avgTaskDuration=36.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533089179_0001_1_00_000000 ], longestDurationTasks=[ task_1463533089179_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:09,400 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:09,400 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089179_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533089179_0001_00_000001, stoppedTime=1463533089400, exitStatus=0
2016-05-18 00:58:09,401 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:09,401 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:09,401 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533089179_0001_1
2016-05-18 00:58:09,401 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089179_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533089179_0001_1
2016-05-18 00:58:09,402 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 6DB8BA4CDB894DC082CB0C8E74AB7A8E for vertex: vertex_1463533089179_0001_1_00 [C8B05AF59A604D8682A94C9622DD9A7F], outputName: 6DB8BA4CDB894DC082CB0C8E74AB7A8E
2016-05-18 00:58:09,403 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533089179_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:09,403 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533089179_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:09,407 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533089179_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:09,407 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533089179_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:09,407 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533089179_0001_00_000001, Exiting
2016-05-18 00:58:09,407 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533089179_0001_00_000001
2016-05-18 00:58:09,408 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533089179_0001_00_000001
2016-05-18 00:58:09,408 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533089179_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:09,409 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:6DB8BA4CDB894DC082CB0C8E74AB7A8E of vertex/vertexGroup:C8B05AF59A604D8682A94C9622DD9A7F isVertexGroupOutput:false
2016-05-18 00:58:09,409 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:09,410 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533089179_0001_00_000001 completed successfully
2016-05-18 00:58:09,411 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533089179_0001_1, queueSize=0
2016-05-18 00:58:09,411 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089179_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533089179_0001_1, startTime=1463533089339, finishTime=1463533089409, timeTaken=70, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=110, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533089390, Process_Duration=4, Process_End_Time=1463533089394, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:09,411 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533089179_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:09,411 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533089179_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:09 Completed Dag: dag_1463533089179_0001_1
2016-05-18 00:58:09,412 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533089179_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:09,412 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533089179_0001_1_post
2016-05-18 00:58:09,412 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:09,412 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:09,412 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:09,413 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[190F25D924E04B14AD0AAEA7374A8CC8/B96B7FC7A7EC45CA9306149E5B34F5B1] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL, with id=dag_1463533089179_0001_1
2016-05-18 00:58:09,413 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[190F25D924E04B14AD0AAEA7374A8CC8/B96B7FC7A7EC45CA9306149E5B34F5B1] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL, with id=dag_1463533089179_0001_1
2016-05-18 00:58:09,420 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...d'_RESULTS.0_'field2'_ALL, applicationId=application_1463533089179_0001
2016-05-18 00:58:09,422 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:09,422 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:09,422 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:09,422 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:09,422 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:09,423 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:09,424 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:09,424 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:09,425 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:09,425 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:09,425 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:09,425 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:09,425 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:09,426 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:09,429 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/190F25D924E04B14AD0AAEA7374A8CC8/B96B7FC7A7EC45CA9306149E5B34F5B1/.tez/application_1463533089179_0001
2016-05-18 00:58:09,429 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_ALL/_temporary
2016-05-18 00:58:09,430 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.287
2016-05-18 00:58:09,499 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_'field2'_ALL
2016-05-18 00:58:09,524 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:09,524 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:09,525 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:58:09,525 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:58:09,557 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-18 00:58:09,564 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-18 00:58:09,565 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:09,565 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:09,565 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:09,582 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:09,582 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_-1
2016-05-18 00:58:09,591 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:09,592 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:09,592 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_-1"]
2016-05-18 00:58:09,592 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:09,592 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:09,592 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:09,593 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...eld'_RESULTS.0_UNKNOWN_-1
2016-05-18 00:58:09,594 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:09,607 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:09,611 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:09,612 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:09,612 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:09,621 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/283C7AB452BA4E4F9D9EFA971E478153/DA0585DC84FF474A92B816113DEE9703/.tez/application_1463533089611_0001 doesn't exist and is created
2016-05-18 00:58:09,634 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:09,634 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:09,635 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/283C7AB452BA4E4F9D9EFA971E478153/DA0585DC84FF474A92B816113DEE9703/.tez/application_1463533089611_0001_wd
2016-05-18 00:58:09,645 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533089611_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:09,645 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:09,645 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:09,647 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:09,647 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:09,647 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:09,647 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:09,648 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:09,649 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:09,649 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:09,649 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:09,649 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:09,649 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:09,649 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:09,649 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:09,650 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:09,650 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:09,650 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:09,650 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:09,650 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:09,651 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:09,652 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:09,652 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:09,652 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/283C7AB452BA4E4F9D9EFA971E478153/DA0585DC84FF474A92B816113DEE9703/.tez/application_1463533089611_0001_wd/localmode-log-dir
2016-05-18 00:58:09,653 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/283C7AB452BA4E4F9D9EFA971E478153/DA0585DC84FF474A92B816113DEE9703/.tez/application_1463533089611_0001_wd/localmode-log-dir/history.txt.appattempt_1463533089611_0001_000000, maxErrors=10
2016-05-18 00:58:09,653 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/283C7AB452BA4E4F9D9EFA971E478153/DA0585DC84FF474A92B816113DEE9703/.tez/application_1463533089611_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:09,653 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533089611_0001_000000, appSubmitTime=1463533089645, launchTime=1463533089645
2016-05-18 00:58:09,657 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:09,664 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:41307
2016-05-18 00:58:09,664 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533089611
2016-05-18 00:58:09,666 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533089611_0001_000000, startTime=1463533089666
2016-05-18 00:58:09,666 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:09,735 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:09,735 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:09,735 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.0_UNKNOWN_-1, applicationId=application_1463533089611_0001, dagName=[283C7AB452BA4E4F9D9EFA971E478153/DA0585DC84FF474A92B816113DEE9703] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1
2016-05-18 00:58:09,751 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [283C7AB452BA4E4F9D9EFA971E478153/DA0585DC84FF474A92B816113DEE9703] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1
2016-05-18 00:58:09,752 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:09,752 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533089611_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/283C7AB452BA4E4F9D9EFA971E478153/DA0585DC84FF474A92B816113DEE9703/.tez/application_1463533089611_0001_wd/localmode-log-dir/dag_1463533089611_0001_1.dot
2016-05-18 00:58:09,753 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/283C7AB452BA4E4F9D9EFA971E478153/DA0585DC84FF474A92B816113DEE9703/.tez/application_1463533089611_0001_wd/localmode-log-dir/dag_1463533089611_0001_1-tez-dag.pb.txt
2016-05-18 00:58:09,756 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533089611_0001_1
2016-05-18 00:58:09,756 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:09,756 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [283C7AB452BA4E4F9D9EFA971E478153/DA0585DC84FF474A92B816113DEE9703] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1
2016-05-18 00:58:09 Running Dag: dag_1463533089611_0001_1
2016-05-18 00:58:09,765 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089611_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533089611_0001_1, submitTime=1463533089751
2016-05-18 00:58:09,766 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/283C7AB452BA4E4F9D9EFA971E478153/DA0585DC84FF474A92B816113DEE9703/.tez/application_1463533089611_0001/tez-conf.pb" } size: 36913 timestamp: 1463533089000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:09,766 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21]
2016-05-18 00:58:09,766 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21]
2016-05-18 00:58:09,767 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:09,767 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:09,767 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089611_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533089611_0001_1, initTime=1463533089767
2016-05-18 00:58:09,767 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533089611_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:09,768 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:09,768 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089611_0001_1][Event:DAG_STARTED]: dagID=dag_1463533089611_0001_1, startTime=1463533089768
2016-05-18 00:58:09,768 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533089611_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:09,768 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: F5876226051C47A5B3D6DD54A1F20E21 : {CE84476F4CFC4C6D874F5821B011FE58={InputName=CE84476F4CFC4C6D874F5821B011FE58}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:09,768 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: CE84476F4CFC4C6D874F5821B011FE58, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:09,768 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21]
2016-05-18 00:58:09,768 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21]
2016-05-18 00:58:09,768 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21]
2016-05-18 00:58:09,769 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21]
2016-05-18 00:58:09,769 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:09,769 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: CE84476F4CFC4C6D874F5821B011FE58 on vertex vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21]
2016-05-18 00:58:09,770 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.0_UNKNOWN_-1, applicationId=application_1463533089611_0001, dagName=[283C7AB452BA4E4F9D9EFA971E478153/DA0585DC84FF474A92B816113DEE9703] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1
2016-05-18 00:58:09,774 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input CE84476F4CFC4C6D874F5821B011FE58 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:09,776 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:09,776 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:09,777 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:09,777 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:09,778 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: CE84476F4CFC4C6D874F5821B011FE58 on vertex vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21]
2016-05-18 00:58:09,779 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:09,779 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {CE84476F4CFC4C6D874F5821B011FE58=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:09,780 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21] parallelism set to 1
2016-05-18 00:58:09,780 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:09,781 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21], numAdditionalOutputs=1
2016-05-18 00:58:09,781 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=E856BF4E74DB41AFBAECB03810EF2506, vertex=vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:09,782 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for F5876226051C47A5B3D6DD54A1F20E21:E856BF4E74DB41AFBAECB03810EF2506 using oldmapred API
2016-05-18 00:58:09,782 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=E856BF4E74DB41AFBAECB03810EF2506, vertexName=F5876226051C47A5B3D6DD54A1F20E21, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:09,783 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089611_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F5876226051C47A5B3D6DD54A1F20E21, vertexId=vertex_1463533089611_0001_1_00, initRequestedTime=1463533089768, initedTime=1463533089781, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:09,783 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21] with distanceFromRoot: 0
2016-05-18 00:58:09,783 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:09,783 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089611_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533089611_0001_1_00, startRequestedTime=1463533089769, startedTime=1463533089783
2016-05-18 00:58:09,783 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089611_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533089611_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:09,783 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:09,784 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in F5876226051C47A5B3D6DD54A1F20E21
2016-05-18 00:58:09,784 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21]
2016-05-18 00:58:09,784 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089611_0001_1][Event:TASK_STARTED]: vertexName=F5876226051C47A5B3D6DD54A1F20E21, taskId=task_1463533089611_0001_1_00_000000, scheduledTime=1463533089784, launchTime=1463533089784
2016-05-18 00:58:09,785 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21]
2016-05-18 00:58:09,785 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:09,787 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533089611_0001_00_000001, launchTime=1463533089787
2016-05-18 00:58:09,787 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533089611_0001_00_000001
2016-05-18 00:58:09,788 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533089611_0001_1_00_000000_0] started. Is using containerId: [container_1463533089611_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:09,788 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089611_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F5876226051C47A5B3D6DD54A1F20E21, taskAttemptId=attempt_1463533089611_0001_1_00_000000_0, startTime=1463533089788, containerId=container_1463533089611_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:09,789 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533089611_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533089611_0001_1_00_000000_0
2016-05-18 00:58:09 Starting to run new task attempt: attempt_1463533089611_0001_1_00_000000_0
2016-05-18 00:58:09,790 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533089611_0001_1_00_000000_0
2016-05-18 00:58:09,791 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:09,791 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:09,791 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [283C7AB452BA4E4F9D9EFA971E478153/DA0585DC84FF474A92B816113DEE9703] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1, VertexName: F5876226051C47A5B3D6DD54A1F20E21, VertexParallelism: 1, TaskAttemptID:attempt_1463533089611_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CE84476F4CFC4C6D874F5821B011FE58, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E856BF4E74DB41AFBAECB03810EF2506, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:09,792 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:09,792 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533089611_0001_1_00_000000_0
2016-05-18 00:58:09,794 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533089611_0001, with dag id: dag_1463533089611_0001_1
2016-05-18 00:58:09,794 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533089611_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:09,795 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533089611_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:09,796 INFO input.MRInput (MRInput.java:initialize(440)) - CE84476F4CFC4C6D874F5821B011FE58 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:09,796 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: CE84476F4CFC4C6D874F5821B011FE58
2016-05-18 00:58:09,799 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:09,808 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F5876226051C47A5B3D6DD54A1F20E21, ordinal: 0
2016-05-18 00:58:09,809 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F5876226051C47A5B3D6DD54A1F20E21, mem on start (mb), free: 197, total: 381, max: 672
2016-05-18 00:58:09,809 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:09,809 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:09,809 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:09,811 INFO output.MROutput (MROutput.java:initialize(421)) - E856BF4E74DB41AFBAECB03810EF2506: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:09,811 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:09,811 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[CE84476F4CFC4C6D874F5821B011FE58:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [E856BF4E74DB41AFBAECB03810EF2506:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:09,812 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:09,812 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:09,812 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:09,813 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[CE84476F4CFC4C6D874F5821B011FE58:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [E856BF4E74DB41AFBAECB03810EF2506:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:09,813 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:09,814 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: CE84476F4CFC4C6D874F5821B011FE58 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:09,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:09,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:09,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: CE84476F4CFC4C6D874F5821B011FE58
2016-05-18 00:58:09,817 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:09,817 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533089611_0001_1_00_000000_0
2016-05-18 00:58:09,818 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:09,818 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - CE84476F4CFC4C6D874F5821B011FE58 initialized RecordReader from event
2016-05-18 00:58:09,820 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:09,821 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: CE84476F4CFC4C6D874F5821B011FE58
2016-05-18 00:58:09,821 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_-1"], id: E856BF4E74DB41AFBAECB03810EF2506
2016-05-18 00:58:09,821 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_-1"]
2016-05-18 00:58:09,822 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:09,822 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F5876226051C47A5B3D6DD54A1F20E21, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:09,825 INFO output.MROutput (MROutput.java:close(526)) - E856BF4E74DB41AFBAECB03810EF2506 closed
2016-05-18 00:58:09,825 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533089611_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:09,826 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533089611_0001_1_00_000000_0
2016-05-18 00:58:09,826 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F5876226051C47A5B3D6DD54A1F20E21, mem on close (mb), free: 195, total: 381, max: 672
2016-05-18 00:58:09,826 INFO output.MROutput (MROutput.java:close(526)) - E856BF4E74DB41AFBAECB03810EF2506 closed
2016-05-18 00:58:09,826 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:09 Completed running task attempt: attempt_1463533089611_0001_1_00_000000_0
2016-05-18 00:58:09,827 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533089611_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:09,827 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533089611_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:09,828 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533089611_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533089822, Process_Duration=4, Process_End_Time=1463533089826, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:09,829 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533089611_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:09,829 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:09,829 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:09,829 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:09,829 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089611_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F5876226051C47A5B3D6DD54A1F20E21, taskAttemptId=attempt_1463533089611_0001_1_00_000000_0, creationTime=1463533089784, allocationTime=1463533089786, startTime=1463533089788, finishTime=1463533089829, timeTaken=41, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:09,829 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533089611_0001_00_000001
2016-05-18 00:58:09,829 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533089611_0001_1_00_000000_0
2016-05-18 00:58:09,831 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089611_0001_1][Event:TASK_FINISHED]: vertexName=F5876226051C47A5B3D6DD54A1F20E21, taskId=task_1463533089611_0001_1_00_000000, startTime=1463533089788, finishTime=1463533089830, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1463533089611_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533089822, Process_Duration=4, Process_End_Time=1463533089826, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:09,831 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:09,831 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21]
2016-05-18 00:58:09,832 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089611_0001_1][Event:VERTEX_FINISHED]: vertexName=F5876226051C47A5B3D6DD54A1F20E21, vertexId=vertex_1463533089611_0001_1_00, initRequestedTime=1463533089768, initedTime=1463533089781, startRequestedTime=1463533089769, startedTime=1463533089783, finishTime=1463533089831, timeTaken=48, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533089822, Process_Duration=4, Process_End_Time=1463533089826, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533089788, firstTasksToStart=[ task_1463533089611_0001_1_00_000000 ], lastTaskFinishTime=1463533089829, lastTasksToFinish=[ task_1463533089611_0001_1_00_000000 ], minTaskDuration=41, maxTaskDuration=41, avgTaskDuration=41.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533089611_0001_1_00_000000 ], longestDurationTasks=[ task_1463533089611_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:09,832 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:09,832 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089611_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533089611_0001_00_000001, stoppedTime=1463533089832, exitStatus=0
2016-05-18 00:58:09,832 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:09,832 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:09,833 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533089611_0001_1
2016-05-18 00:58:09,833 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089611_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533089611_0001_1
2016-05-18 00:58:09,833 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: E856BF4E74DB41AFBAECB03810EF2506 for vertex: vertex_1463533089611_0001_1_00 [F5876226051C47A5B3D6DD54A1F20E21], outputName: E856BF4E74DB41AFBAECB03810EF2506
2016-05-18 00:58:09,834 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533089611_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:09,835 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533089611_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:09,839 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:E856BF4E74DB41AFBAECB03810EF2506 of vertex/vertexGroup:F5876226051C47A5B3D6DD54A1F20E21 isVertexGroupOutput:false
2016-05-18 00:58:09,839 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:09,840 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533089611_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:09,841 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533089611_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:09,841 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533089611_0001_1, queueSize=0
2016-05-18 00:58:09,841 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533089611_0001_00_000001, Exiting
2016-05-18 00:58:09,841 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533089611_0001_00_000001
2016-05-18 00:58:09,841 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533089611_0001_00_000001
2016-05-18 00:58:09,842 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533089611_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533089611_0001_1, startTime=1463533089768, finishTime=1463533089839, timeTaken=71, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=130, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533089822, Process_Duration=4, Process_End_Time=1463533089826, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:09,842 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533089611_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:09,842 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533089611_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:09 Completed Dag: dag_1463533089611_0001_1
2016-05-18 00:58:09,843 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533089611_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:09,843 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533089611_0001_1_post
2016-05-18 00:58:09,843 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:09,843 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:09,843 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:09,843 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[283C7AB452BA4E4F9D9EFA971E478153/DA0585DC84FF474A92B816113DEE9703] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1, with id=dag_1463533089611_0001_1
2016-05-18 00:58:09,843 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533089611_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:09,844 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533089611_0001_00_000001 completed successfully
2016-05-18 00:58:09,844 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[283C7AB452BA4E4F9D9EFA971E478153/DA0585DC84FF474A92B816113DEE9703] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1, with id=dag_1463533089611_0001_1
2016-05-18 00:58:09,849 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...eld'_RESULTS.0_UNKNOWN_-1, applicationId=application_1463533089611_0001
2016-05-18 00:58:09,851 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:09,851 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:09,851 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:09,851 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:09,851 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:09,852 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:09,852 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:09,852 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:09,852 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:09,852 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:09,853 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:09,853 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:09,853 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:09,853 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:09,858 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/283C7AB452BA4E4F9D9EFA971E478153/DA0585DC84FF474A92B816113DEE9703/.tez/application_1463533089611_0001
2016-05-18 00:58:09,859 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_-1/_temporary
2016-05-18 00:58:09,859 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.267
2016-05-18 00:58:09,927 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_UNKNOWN_-1
2016-05-18 00:58:09,951 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:09,951 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:09,981 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-18 00:58:09,985 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.007
2016-05-18 00:58:09,985 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 00:58:09,985 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 00:58:09,986 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_UNKNOWN_'field2'
2016-05-18 00:58:10,010 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:10,011 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:10,046 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-05-18 00:58:10,047 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:58:10,047 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:58:10,052 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-18 00:58:10,052 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:10,052 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:10,053 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:10,068 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:10,069 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_RESULTS
2016-05-18 00:58:10,078 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:10,078 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:10,078 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_RESULTS"]
2016-05-18 00:58:10,078 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:10,078 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:10,079 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:10,080 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...RESULTS.0_UNKNOWN_RESULTS
2016-05-18 00:58:10,081 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:10,093 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:10,098 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:10,099 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:10,099 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:10,110 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/BA3B41F62B1A4CDEB7DEC664D4ACB05B/E4259D627D0945BE8595D6A21E9A9F59/.tez/application_1463533090099_0001 doesn't exist and is created
2016-05-18 00:58:10,124 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:10,124 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:10,125 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/BA3B41F62B1A4CDEB7DEC664D4ACB05B/E4259D627D0945BE8595D6A21E9A9F59/.tez/application_1463533090099_0001_wd
2016-05-18 00:58:10,136 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533090099_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:10,136 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:10,137 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:10,138 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:10,138 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:10,138 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:10,138 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:10,139 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:10,140 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:10,140 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:10,141 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:10,141 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:10,141 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:10,141 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:10,141 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:10,141 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:10,142 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:10,142 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:10,142 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:10,142 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:10,143 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:10,144 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:10,144 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:10,144 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/BA3B41F62B1A4CDEB7DEC664D4ACB05B/E4259D627D0945BE8595D6A21E9A9F59/.tez/application_1463533090099_0001_wd/localmode-log-dir
2016-05-18 00:58:10,147 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/BA3B41F62B1A4CDEB7DEC664D4ACB05B/E4259D627D0945BE8595D6A21E9A9F59/.tez/application_1463533090099_0001_wd/localmode-log-dir/history.txt.appattempt_1463533090099_0001_000000, maxErrors=10
2016-05-18 00:58:10,147 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/BA3B41F62B1A4CDEB7DEC664D4ACB05B/E4259D627D0945BE8595D6A21E9A9F59/.tez/application_1463533090099_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:10,147 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533090099_0001_000000, appSubmitTime=1463533090136, launchTime=1463533090136
2016-05-18 00:58:10,154 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:10,172 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:35093
2016-05-18 00:58:10,172 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533090099
2016-05-18 00:58:10,174 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533090099_0001_000000, startTime=1463533090174
2016-05-18 00:58:10,174 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:10,225 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:10,225 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:10,225 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...RESULTS.0_UNKNOWN_RESULTS, applicationId=application_1463533090099_0001, dagName=[BA3B41F62B1A4CDEB7DEC664D4ACB05B/E4259D627D0945BE8595D6A21E9A9F59] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS
2016-05-18 00:58:10,241 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [BA3B41F62B1A4CDEB7DEC664D4ACB05B/E4259D627D0945BE8595D6A21E9A9F59] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS
2016-05-18 00:58:10,241 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:10,242 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533090099_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/BA3B41F62B1A4CDEB7DEC664D4ACB05B/E4259D627D0945BE8595D6A21E9A9F59/.tez/application_1463533090099_0001_wd/localmode-log-dir/dag_1463533090099_0001_1.dot
2016-05-18 00:58:10,243 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/BA3B41F62B1A4CDEB7DEC664D4ACB05B/E4259D627D0945BE8595D6A21E9A9F59/.tez/application_1463533090099_0001_wd/localmode-log-dir/dag_1463533090099_0001_1-tez-dag.pb.txt
2016-05-18 00:58:10,246 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533090099_0001_1
2016-05-18 00:58:10,246 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:10,246 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [BA3B41F62B1A4CDEB7DEC664D4ACB05B/E4259D627D0945BE8595D6A21E9A9F59] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS
2016-05-18 00:58:10 Running Dag: dag_1463533090099_0001_1
2016-05-18 00:58:10,255 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090099_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533090099_0001_1, submitTime=1463533090241
2016-05-18 00:58:10,256 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/BA3B41F62B1A4CDEB7DEC664D4ACB05B/E4259D627D0945BE8595D6A21E9A9F59/.tez/application_1463533090099_0001/tez-conf.pb" } size: 36913 timestamp: 1463533090000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:10,256 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72]
2016-05-18 00:58:10,256 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72]
2016-05-18 00:58:10,258 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:10,258 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:10,258 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090099_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533090099_0001_1, initTime=1463533090258
2016-05-18 00:58:10,259 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533090099_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:10,260 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:10,260 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090099_0001_1][Event:DAG_STARTED]: dagID=dag_1463533090099_0001_1, startTime=1463533090260
2016-05-18 00:58:10,260 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533090099_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:10,260 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: F2D4FFC6E104429BB73A583D0688FB72 : {18C55189E33A4E73A4514ADED206D193={InputName=18C55189E33A4E73A4514ADED206D193}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:10,260 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 18C55189E33A4E73A4514ADED206D193, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:10,260 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72]
2016-05-18 00:58:10,260 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72]
2016-05-18 00:58:10,260 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72]
2016-05-18 00:58:10,261 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72]
2016-05-18 00:58:10,261 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:10,262 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 18C55189E33A4E73A4514ADED206D193 on vertex vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72]
2016-05-18 00:58:10,263 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...RESULTS.0_UNKNOWN_RESULTS, applicationId=application_1463533090099_0001, dagName=[BA3B41F62B1A4CDEB7DEC664D4ACB05B/E4259D627D0945BE8595D6A21E9A9F59] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS
2016-05-18 00:58:10,265 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 18C55189E33A4E73A4514ADED206D193 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:10,267 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:10,267 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:10,268 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:10,268 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:10,269 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 18C55189E33A4E73A4514ADED206D193 on vertex vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72]
2016-05-18 00:58:10,270 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:10,270 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {18C55189E33A4E73A4514ADED206D193=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:10,271 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72] parallelism set to 1
2016-05-18 00:58:10,271 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:10,272 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72], numAdditionalOutputs=1
2016-05-18 00:58:10,272 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=95D17E1028CB4D419947DD86FCDED48E, vertex=vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:10,274 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for F2D4FFC6E104429BB73A583D0688FB72:95D17E1028CB4D419947DD86FCDED48E using oldmapred API
2016-05-18 00:58:10,274 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=95D17E1028CB4D419947DD86FCDED48E, vertexName=F2D4FFC6E104429BB73A583D0688FB72, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:10,276 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090099_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F2D4FFC6E104429BB73A583D0688FB72, vertexId=vertex_1463533090099_0001_1_00, initRequestedTime=1463533090260, initedTime=1463533090272, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:10,276 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72] with distanceFromRoot: 0
2016-05-18 00:58:10,276 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:10,276 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090099_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533090099_0001_1_00, startRequestedTime=1463533090261, startedTime=1463533090276
2016-05-18 00:58:10,277 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090099_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533090099_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:10,277 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:10,277 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in F2D4FFC6E104429BB73A583D0688FB72
2016-05-18 00:58:10,277 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72]
2016-05-18 00:58:10,278 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090099_0001_1][Event:TASK_STARTED]: vertexName=F2D4FFC6E104429BB73A583D0688FB72, taskId=task_1463533090099_0001_1_00_000000, scheduledTime=1463533090278, launchTime=1463533090278
2016-05-18 00:58:10,279 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72]
2016-05-18 00:58:10,280 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:10,281 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533090099_0001_00_000001, launchTime=1463533090281
2016-05-18 00:58:10,282 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533090099_0001, with dag id: dag_1463533090099_0001_1
2016-05-18 00:58:10,283 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533090099_0001_00_000001
2016-05-18 00:58:10,286 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533090099_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533090099_0001_1_00_000000_0
2016-05-18 00:58:10,286 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533090099_0001_1_00_000000_0] started. Is using containerId: [container_1463533090099_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:10,286 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090099_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F2D4FFC6E104429BB73A583D0688FB72, taskAttemptId=attempt_1463533090099_0001_1_00_000000_0, startTime=1463533090286, containerId=container_1463533090099_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:10 Starting to run new task attempt: attempt_1463533090099_0001_1_00_000000_0
2016-05-18 00:58:10,286 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533090099_0001_1_00_000000_0
2016-05-18 00:58:10,286 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:10,286 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:10,287 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BA3B41F62B1A4CDEB7DEC664D4ACB05B/E4259D627D0945BE8595D6A21E9A9F59] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS, VertexName: F2D4FFC6E104429BB73A583D0688FB72, VertexParallelism: 1, TaskAttemptID:attempt_1463533090099_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=18C55189E33A4E73A4514ADED206D193, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=95D17E1028CB4D419947DD86FCDED48E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:10,288 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:10,289 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533090099_0001_1_00_000000_0
2016-05-18 00:58:10,290 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533090099_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:10,290 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533090099_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:10,293 INFO input.MRInput (MRInput.java:initialize(440)) - 18C55189E33A4E73A4514ADED206D193 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:10,293 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 18C55189E33A4E73A4514ADED206D193
2016-05-18 00:58:10,293 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:10,299 INFO output.MROutput (MROutput.java:initialize(421)) - 95D17E1028CB4D419947DD86FCDED48E: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:10,300 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F2D4FFC6E104429BB73A583D0688FB72, ordinal: 0
2016-05-18 00:58:10,300 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F2D4FFC6E104429BB73A583D0688FB72, mem on start (mb), free: 293, total: 383, max: 672
2016-05-18 00:58:10,300 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:10,301 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:10,301 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:10,301 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:10,301 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[18C55189E33A4E73A4514ADED206D193:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [95D17E1028CB4D419947DD86FCDED48E:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:10,302 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:10,302 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:10,302 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:10,303 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[18C55189E33A4E73A4514ADED206D193:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [95D17E1028CB4D419947DD86FCDED48E:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:10,303 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:10,303 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 18C55189E33A4E73A4514ADED206D193 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:10,303 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:10,304 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:10,303 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 18C55189E33A4E73A4514ADED206D193
2016-05-18 00:58:10,304 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:10,304 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533090099_0001_1_00_000000_0
2016-05-18 00:58:10,306 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:10,306 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 18C55189E33A4E73A4514ADED206D193 initialized RecordReader from event
2016-05-18 00:58:10,307 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:10,308 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 18C55189E33A4E73A4514ADED206D193
2016-05-18 00:58:10,308 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_RESULTS"], id: 95D17E1028CB4D419947DD86FCDED48E
2016-05-18 00:58:10,308 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_RESULTS"]
2016-05-18 00:58:10,309 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:10,309 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F2D4FFC6E104429BB73A583D0688FB72, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:10,312 INFO output.MROutput (MROutput.java:close(526)) - 95D17E1028CB4D419947DD86FCDED48E closed
2016-05-18 00:58:10,312 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533090099_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:10,313 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533090099_0001_1_00_000000_0
2016-05-18 00:58:10,313 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F2D4FFC6E104429BB73A583D0688FB72, mem on close (mb), free: 291, total: 383, max: 672
2016-05-18 00:58:10,313 INFO output.MROutput (MROutput.java:close(526)) - 95D17E1028CB4D419947DD86FCDED48E closed
2016-05-18 00:58:10,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:10 Completed running task attempt: attempt_1463533090099_0001_1_00_000000_0
2016-05-18 00:58:10,314 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533090099_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:10,314 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533090099_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:10,315 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533090099_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533090309, Process_Duration=3, Process_End_Time=1463533090312, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:10,316 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533090099_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:10,316 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:10,316 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090099_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F2D4FFC6E104429BB73A583D0688FB72, taskAttemptId=attempt_1463533090099_0001_1_00_000000_0, creationTime=1463533090279, allocationTime=1463533090281, startTime=1463533090286, finishTime=1463533090316, timeTaken=30, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:10,316 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533090099_0001_1_00_000000_0
2016-05-18 00:58:10,316 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:10,317 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:10,318 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533090099_0001_00_000001
2016-05-18 00:58:10,317 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090099_0001_1][Event:TASK_FINISHED]: vertexName=F2D4FFC6E104429BB73A583D0688FB72, taskId=task_1463533090099_0001_1_00_000000, startTime=1463533090286, finishTime=1463533090317, timeTaken=31, status=SUCCEEDED, successfulAttemptID=attempt_1463533090099_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533090309, Process_Duration=3, Process_End_Time=1463533090312, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:10,318 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:10,318 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72]
2016-05-18 00:58:10,320 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090099_0001_1][Event:VERTEX_FINISHED]: vertexName=F2D4FFC6E104429BB73A583D0688FB72, vertexId=vertex_1463533090099_0001_1_00, initRequestedTime=1463533090260, initedTime=1463533090272, startRequestedTime=1463533090261, startedTime=1463533090276, finishTime=1463533090318, timeTaken=42, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533090309, Process_Duration=3, Process_End_Time=1463533090312, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533090286, firstTasksToStart=[ task_1463533090099_0001_1_00_000000 ], lastTaskFinishTime=1463533090316, lastTasksToFinish=[ task_1463533090099_0001_1_00_000000 ], minTaskDuration=30, maxTaskDuration=30, avgTaskDuration=30.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533090099_0001_1_00_000000 ], longestDurationTasks=[ task_1463533090099_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:10,320 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:10,320 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090099_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533090099_0001_00_000001, stoppedTime=1463533090320, exitStatus=0
2016-05-18 00:58:10,320 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:10,321 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:10,321 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533090099_0001_1
2016-05-18 00:58:10,321 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090099_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533090099_0001_1
2016-05-18 00:58:10,321 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533090099_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:10,321 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 95D17E1028CB4D419947DD86FCDED48E for vertex: vertex_1463533090099_0001_1_00 [F2D4FFC6E104429BB73A583D0688FB72], outputName: 95D17E1028CB4D419947DD86FCDED48E
2016-05-18 00:58:10,321 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533090099_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:10,327 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:95D17E1028CB4D419947DD86FCDED48E of vertex/vertexGroup:F2D4FFC6E104429BB73A583D0688FB72 isVertexGroupOutput:false
2016-05-18 00:58:10,327 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:10,328 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533090099_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:10,328 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533090099_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:10,328 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533090099_0001_00_000001, Exiting
2016-05-18 00:58:10,329 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533090099_0001_00_000001
2016-05-18 00:58:10,329 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533090099_0001_00_000001
2016-05-18 00:58:10,329 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533090099_0001_1, queueSize=0
2016-05-18 00:58:10,330 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090099_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533090099_0001_1, startTime=1463533090260, finishTime=1463533090327, timeTaken=67, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=110, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533090309, Process_Duration=3, Process_End_Time=1463533090312, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:10,329 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533090099_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:10,330 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533090099_0001_00_000001 completed successfully
2016-05-18 00:58:10,330 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533090099_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:10,331 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533090099_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:10 Completed Dag: dag_1463533090099_0001_1
2016-05-18 00:58:10,331 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533090099_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:10,332 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533090099_0001_1_post
2016-05-18 00:58:10,332 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:10,332 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 1
2016-05-18 00:58:10,332 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:10,333 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[BA3B41F62B1A4CDEB7DEC664D4ACB05B/E4259D627D0945BE8595D6A21E9A9F59] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS, with id=dag_1463533090099_0001_1
2016-05-18 00:58:10,333 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[BA3B41F62B1A4CDEB7DEC664D4ACB05B/E4259D627D0945BE8595D6A21E9A9F59] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS, with id=dag_1463533090099_0001_1
2016-05-18 00:58:10,337 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...RESULTS.0_UNKNOWN_RESULTS, applicationId=application_1463533090099_0001
2016-05-18 00:58:10,339 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:10,339 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:10,339 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:10,339 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:10,339 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:10,340 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:10,340 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:10,340 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:10,341 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:10,341 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:10,341 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:10,342 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:10,342 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:10,343 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:10,346 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/BA3B41F62B1A4CDEB7DEC664D4ACB05B/E4259D627D0945BE8595D6A21E9A9F59/.tez/application_1463533090099_0001
2016-05-18 00:58:10,347 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_RESULTS/_temporary
2016-05-18 00:58:10,347 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.268
2016-05-18 00:58:10,422 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_UNKNOWN_RESULTS
2016-05-18 00:58:10,446 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:10,446 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:10,481 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-18 00:58:10,493 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-05-18 00:58:10,493 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:10,494 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:10,494 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:10,510 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:10,510 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_ALL
2016-05-18 00:58:10,521 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:10,521 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:10,521 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_ALL"]
2016-05-18 00:58:10,521 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:10,521 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:10,521 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:10,522 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ld'_RESULTS.0_UNKNOWN_ALL
2016-05-18 00:58:10,522 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:10,523 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:58:10,523 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:58:10,535 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:10,539 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:10,540 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:10,540 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:10,548 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/5521E612C50F4B5B93ED786286BCBA02/FC49B301C4D243E3900F16D77FBB834E/.tez/application_1463533090539_0001 doesn't exist and is created
2016-05-18 00:58:10,563 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:10,564 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:10,564 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/5521E612C50F4B5B93ED786286BCBA02/FC49B301C4D243E3900F16D77FBB834E/.tez/application_1463533090539_0001_wd
2016-05-18 00:58:10,573 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533090539_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:10,573 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:10,574 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:10,575 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:10,575 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:10,575 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:10,575 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:10,576 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:10,577 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:10,577 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:10,577 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:10,577 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:10,577 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:10,577 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:10,577 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:10,577 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:10,577 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:10,577 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:10,578 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:10,578 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:10,579 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:10,580 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:10,580 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:10,581 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/5521E612C50F4B5B93ED786286BCBA02/FC49B301C4D243E3900F16D77FBB834E/.tez/application_1463533090539_0001_wd/localmode-log-dir
2016-05-18 00:58:10,581 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/5521E612C50F4B5B93ED786286BCBA02/FC49B301C4D243E3900F16D77FBB834E/.tez/application_1463533090539_0001_wd/localmode-log-dir/history.txt.appattempt_1463533090539_0001_000000, maxErrors=10
2016-05-18 00:58:10,582 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/5521E612C50F4B5B93ED786286BCBA02/FC49B301C4D243E3900F16D77FBB834E/.tez/application_1463533090539_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:10,582 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533090539_0001_000000, appSubmitTime=1463533090573, launchTime=1463533090573
2016-05-18 00:58:10,586 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:10,593 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:58483
2016-05-18 00:58:10,594 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533090539
2016-05-18 00:58:10,595 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533090539_0001_000000, startTime=1463533090595
2016-05-18 00:58:10,595 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:10,664 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:10,664 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:10,664 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL, applicationId=application_1463533090539_0001, dagName=[5521E612C50F4B5B93ED786286BCBA02/FC49B301C4D243E3900F16D77FBB834E] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL
2016-05-18 00:58:10,681 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [5521E612C50F4B5B93ED786286BCBA02/FC49B301C4D243E3900F16D77FBB834E] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL
2016-05-18 00:58:10,681 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:10,682 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533090539_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/5521E612C50F4B5B93ED786286BCBA02/FC49B301C4D243E3900F16D77FBB834E/.tez/application_1463533090539_0001_wd/localmode-log-dir/dag_1463533090539_0001_1.dot
2016-05-18 00:58:10,682 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/5521E612C50F4B5B93ED786286BCBA02/FC49B301C4D243E3900F16D77FBB834E/.tez/application_1463533090539_0001_wd/localmode-log-dir/dag_1463533090539_0001_1-tez-dag.pb.txt
2016-05-18 00:58:10,686 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533090539_0001_1
2016-05-18 00:58:10,686 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:10,686 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [5521E612C50F4B5B93ED786286BCBA02/FC49B301C4D243E3900F16D77FBB834E] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL
2016-05-18 00:58:10 Running Dag: dag_1463533090539_0001_1
2016-05-18 00:58:10,695 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090539_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533090539_0001_1, submitTime=1463533090681
2016-05-18 00:58:10,696 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/5521E612C50F4B5B93ED786286BCBA02/FC49B301C4D243E3900F16D77FBB834E/.tez/application_1463533090539_0001/tez-conf.pb" } size: 36913 timestamp: 1463533090000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:10,697 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED]
2016-05-18 00:58:10,697 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED]
2016-05-18 00:58:10,697 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:10,697 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:10,698 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090539_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533090539_0001_1, initTime=1463533090697
2016-05-18 00:58:10,698 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533090539_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:10,698 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:10,699 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL, applicationId=application_1463533090539_0001, dagName=[5521E612C50F4B5B93ED786286BCBA02/FC49B301C4D243E3900F16D77FBB834E] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL
2016-05-18 00:58:10,699 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090539_0001_1][Event:DAG_STARTED]: dagID=dag_1463533090539_0001_1, startTime=1463533090699
2016-05-18 00:58:10,699 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533090539_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:10,700 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: E96F3F435DB34191BE381E354DAEA5ED : {595AE5CC4E9540309E250CD9E09F350A={InputName=595AE5CC4E9540309E250CD9E09F350A}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:10,700 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 595AE5CC4E9540309E250CD9E09F350A, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:10,700 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED]
2016-05-18 00:58:10,700 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED]
2016-05-18 00:58:10,700 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED]
2016-05-18 00:58:10,700 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED]
2016-05-18 00:58:10,701 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:10,702 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 595AE5CC4E9540309E250CD9E09F350A on vertex vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED]
2016-05-18 00:58:10,705 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 595AE5CC4E9540309E250CD9E09F350A asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:10,707 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:10,707 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:10,708 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:10,708 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:10,709 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 595AE5CC4E9540309E250CD9E09F350A on vertex vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED]
2016-05-18 00:58:10,712 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:10,712 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {595AE5CC4E9540309E250CD9E09F350A=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:10,713 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED] parallelism set to 1
2016-05-18 00:58:10,713 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:10,713 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED], numAdditionalOutputs=1
2016-05-18 00:58:10,713 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=983EF917C5C74D1F9E5B6C457F79127D, vertex=vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:10,715 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for E96F3F435DB34191BE381E354DAEA5ED:983EF917C5C74D1F9E5B6C457F79127D using oldmapred API
2016-05-18 00:58:10,715 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=983EF917C5C74D1F9E5B6C457F79127D, vertexName=E96F3F435DB34191BE381E354DAEA5ED, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:10,716 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090539_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E96F3F435DB34191BE381E354DAEA5ED, vertexId=vertex_1463533090539_0001_1_00, initRequestedTime=1463533090700, initedTime=1463533090713, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:10,716 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED] with distanceFromRoot: 0
2016-05-18 00:58:10,716 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:10,716 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090539_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533090539_0001_1_00, startRequestedTime=1463533090701, startedTime=1463533090716
2016-05-18 00:58:10,717 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090539_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533090539_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:10,717 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:10,717 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in E96F3F435DB34191BE381E354DAEA5ED
2016-05-18 00:58:10,718 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED]
2016-05-18 00:58:10,718 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090539_0001_1][Event:TASK_STARTED]: vertexName=E96F3F435DB34191BE381E354DAEA5ED, taskId=task_1463533090539_0001_1_00_000000, scheduledTime=1463533090718, launchTime=1463533090718
2016-05-18 00:58:10,718 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED]
2016-05-18 00:58:10,719 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:10,721 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533090539_0001_00_000001, launchTime=1463533090721
2016-05-18 00:58:10,722 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533090539_0001_00_000001
2016-05-18 00:58:10,724 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533090539_0001_1_00_000000_0] started. Is using containerId: [container_1463533090539_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:10,724 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090539_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E96F3F435DB34191BE381E354DAEA5ED, taskAttemptId=attempt_1463533090539_0001_1_00_000000_0, startTime=1463533090723, containerId=container_1463533090539_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:10,725 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533090539_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533090539_0001_1_00_000000_0
2016-05-18 00:58:10 Starting to run new task attempt: attempt_1463533090539_0001_1_00_000000_0
2016-05-18 00:58:10,726 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533090539_0001_1_00_000000_0
2016-05-18 00:58:10,726 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:10,726 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:10,727 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5521E612C50F4B5B93ED786286BCBA02/FC49B301C4D243E3900F16D77FBB834E] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL, VertexName: E96F3F435DB34191BE381E354DAEA5ED, VertexParallelism: 1, TaskAttemptID:attempt_1463533090539_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=595AE5CC4E9540309E250CD9E09F350A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=983EF917C5C74D1F9E5B6C457F79127D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:10,727 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:10,727 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533090539_0001_1_00_000000_0
2016-05-18 00:58:10,730 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533090539_0001, with dag id: dag_1463533090539_0001_1
2016-05-18 00:58:10,731 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:10,732 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533090539_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:10,732 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533090539_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:10,740 INFO input.MRInput (MRInput.java:initialize(440)) - 595AE5CC4E9540309E250CD9E09F350A using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:10,740 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 595AE5CC4E9540309E250CD9E09F350A
2016-05-18 00:58:10,741 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E96F3F435DB34191BE381E354DAEA5ED, ordinal: 0
2016-05-18 00:58:10,741 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E96F3F435DB34191BE381E354DAEA5ED, mem on start (mb), free: 207, total: 383, max: 672
2016-05-18 00:58:10,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:10,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:10,742 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:10,744 INFO output.MROutput (MROutput.java:initialize(421)) - 983EF917C5C74D1F9E5B6C457F79127D: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:10,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:10,745 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[983EF917C5C74D1F9E5B6C457F79127D:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [595AE5CC4E9540309E250CD9E09F350A:INPUT:0:org.apache.tez.mapreduce.input.MRInput]
2016-05-18 00:58:10,745 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:10,745 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:10,745 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:10,746 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[983EF917C5C74D1F9E5B6C457F79127D:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [595AE5CC4E9540309E250CD9E09F350A:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0]
2016-05-18 00:58:10,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:10,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 595AE5CC4E9540309E250CD9E09F350A being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:10,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 595AE5CC4E9540309E250CD9E09F350A
2016-05-18 00:58:10,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:10,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:10,747 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:10,747 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533090539_0001_1_00_000000_0
2016-05-18 00:58:10,748 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:10,748 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 595AE5CC4E9540309E250CD9E09F350A initialized RecordReader from event
2016-05-18 00:58:10,750 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:10,750 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 595AE5CC4E9540309E250CD9E09F350A
2016-05-18 00:58:10,750 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_ALL"], id: 983EF917C5C74D1F9E5B6C457F79127D
2016-05-18 00:58:10,751 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_ALL"]
2016-05-18 00:58:10,751 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:10,751 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E96F3F435DB34191BE381E354DAEA5ED, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:10,754 INFO output.MROutput (MROutput.java:close(526)) - 983EF917C5C74D1F9E5B6C457F79127D closed
2016-05-18 00:58:10,754 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533090539_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:10,755 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533090539_0001_1_00_000000_0
2016-05-18 00:58:10,755 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E96F3F435DB34191BE381E354DAEA5ED, mem on close (mb), free: 205, total: 383, max: 672
2016-05-18 00:58:10,755 INFO output.MROutput (MROutput.java:close(526)) - 983EF917C5C74D1F9E5B6C457F79127D closed
2016-05-18 00:58:10,755 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:10 Completed running task attempt: attempt_1463533090539_0001_1_00_000000_0
2016-05-18 00:58:10,756 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533090539_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:10,756 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533090539_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:10,757 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533090539_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533090751, Process_Duration=4, Process_End_Time=1463533090755, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:10,758 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533090539_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:10,758 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:10,758 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090539_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E96F3F435DB34191BE381E354DAEA5ED, taskAttemptId=attempt_1463533090539_0001_1_00_000000_0, creationTime=1463533090718, allocationTime=1463533090720, startTime=1463533090723, finishTime=1463533090758, timeTaken=35, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:10,759 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533090539_0001_1_00_000000_0
2016-05-18 00:58:10,758 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:10,759 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:10,759 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533090539_0001_00_000001
2016-05-18 00:58:10,759 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090539_0001_1][Event:TASK_FINISHED]: vertexName=E96F3F435DB34191BE381E354DAEA5ED, taskId=task_1463533090539_0001_1_00_000000, startTime=1463533090723, finishTime=1463533090759, timeTaken=36, status=SUCCEEDED, successfulAttemptID=attempt_1463533090539_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533090751, Process_Duration=4, Process_End_Time=1463533090755, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:10,760 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:10,760 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED]
2016-05-18 00:58:10,761 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090539_0001_1][Event:VERTEX_FINISHED]: vertexName=E96F3F435DB34191BE381E354DAEA5ED, vertexId=vertex_1463533090539_0001_1_00, initRequestedTime=1463533090700, initedTime=1463533090713, startRequestedTime=1463533090701, startedTime=1463533090716, finishTime=1463533090760, timeTaken=44, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533090751, Process_Duration=4, Process_End_Time=1463533090755, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533090723, firstTasksToStart=[ task_1463533090539_0001_1_00_000000 ], lastTaskFinishTime=1463533090758, lastTasksToFinish=[ task_1463533090539_0001_1_00_000000 ], minTaskDuration=35, maxTaskDuration=35, avgTaskDuration=35.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533090539_0001_1_00_000000 ], longestDurationTasks=[ task_1463533090539_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:10,761 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:10,762 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090539_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533090539_0001_00_000001, stoppedTime=1463533090761, exitStatus=0
2016-05-18 00:58:10,762 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:10,762 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:10,762 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533090539_0001_1
2016-05-18 00:58:10,762 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090539_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533090539_0001_1
2016-05-18 00:58:10,762 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533090539_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:10,762 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533090539_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:10,762 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 983EF917C5C74D1F9E5B6C457F79127D for vertex: vertex_1463533090539_0001_1_00 [E96F3F435DB34191BE381E354DAEA5ED], outputName: 983EF917C5C74D1F9E5B6C457F79127D
2016-05-18 00:58:10,763 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533090539_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:10,763 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533090539_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:10,763 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533090539_0001_00_000001, Exiting
2016-05-18 00:58:10,763 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533090539_0001_00_000001
2016-05-18 00:58:10,763 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533090539_0001_00_000001
2016-05-18 00:58:10,764 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533090539_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:10,764 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533090539_0001_00_000001 completed successfully
2016-05-18 00:58:10,769 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:983EF917C5C74D1F9E5B6C457F79127D of vertex/vertexGroup:E96F3F435DB34191BE381E354DAEA5ED isVertexGroupOutput:false
2016-05-18 00:58:10,769 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:10,771 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533090539_0001_1, queueSize=0
2016-05-18 00:58:10,771 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090539_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533090539_0001_1, startTime=1463533090698, finishTime=1463533090769, timeTaken=71, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=130, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533090751, Process_Duration=4, Process_End_Time=1463533090755, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:10,772 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533090539_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:10,772 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533090539_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:10 Completed Dag: dag_1463533090539_0001_1
2016-05-18 00:58:10,772 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533090539_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:10,772 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533090539_0001_1_post
2016-05-18 00:58:10,772 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:10,772 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:10,773 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:10,773 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[5521E612C50F4B5B93ED786286BCBA02/FC49B301C4D243E3900F16D77FBB834E] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL, with id=dag_1463533090539_0001_1
2016-05-18 00:58:10,773 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[5521E612C50F4B5B93ED786286BCBA02/FC49B301C4D243E3900F16D77FBB834E] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL, with id=dag_1463533090539_0001_1
2016-05-18 00:58:10,783 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL, applicationId=application_1463533090539_0001
2016-05-18 00:58:10,785 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:10,785 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:10,785 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:10,785 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:10,785 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:10,787 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:10,787 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:10,787 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:10,787 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:10,787 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:10,788 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:10,788 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:10,789 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:10,790 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:10,796 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/5521E612C50F4B5B93ED786286BCBA02/FC49B301C4D243E3900F16D77FBB834E/.tez/application_1463533090539_0001
2016-05-18 00:58:10,796 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_ALL/_temporary
2016-05-18 00:58:10,797 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.275
2016-05-18 00:58:10,868 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_UNKNOWN_ALL
2016-05-18 00:58:10,893 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:10,893 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:10,931 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-18 00:58:10,936 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-18 00:58:10,936 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:10,937 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:10,938 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:10,952 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:10,952 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_-1
2016-05-18 00:58:10,962 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:10,962 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:10,962 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_-1"]
2016-05-18 00:58:10,962 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:10,963 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:10,963 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:10,964 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-18 00:58:10,964 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:10,976 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:10,981 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:10,981 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:10,981 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:10,991 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7444B599F3154FAA85AEDFC5A89956A9/DA1FA40C05D541C483DBE4EDA960A628/.tez/application_1463533090981_0001 doesn't exist and is created
2016-05-18 00:58:11,006 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:11,007 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:11,008 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7444B599F3154FAA85AEDFC5A89956A9/DA1FA40C05D541C483DBE4EDA960A628/.tez/application_1463533090981_0001_wd
2016-05-18 00:58:11,019 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533090981_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:11,020 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:11,020 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:11,021 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:11,021 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:11,021 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:11,021 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:11,022 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:11,024 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:11,024 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:11,024 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:11,024 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:11,024 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:11,024 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:11,024 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:11,025 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:11,025 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:11,025 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:11,025 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:11,025 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:11,027 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:11,028 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:11,028 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:11,029 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7444B599F3154FAA85AEDFC5A89956A9/DA1FA40C05D541C483DBE4EDA960A628/.tez/application_1463533090981_0001_wd/localmode-log-dir
2016-05-18 00:58:11,029 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7444B599F3154FAA85AEDFC5A89956A9/DA1FA40C05D541C483DBE4EDA960A628/.tez/application_1463533090981_0001_wd/localmode-log-dir/history.txt.appattempt_1463533090981_0001_000000, maxErrors=10
2016-05-18 00:58:11,029 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7444B599F3154FAA85AEDFC5A89956A9/DA1FA40C05D541C483DBE4EDA960A628/.tez/application_1463533090981_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:11,030 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533090981_0001_000000, appSubmitTime=1463533091019, launchTime=1463533091019
2016-05-18 00:58:11,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:11,040 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:58:11,040 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:58:11,042 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:37827
2016-05-18 00:58:11,043 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533090981
2016-05-18 00:58:11,044 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533090981_0001_000000, startTime=1463533091044
2016-05-18 00:58:11,044 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:11,107 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:11,107 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:11,107 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ield'_RESULTS.0_VALUES_-1, applicationId=application_1463533090981_0001, dagName=[7444B599F3154FAA85AEDFC5A89956A9/DA1FA40C05D541C483DBE4EDA960A628] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-18 00:58:11,123 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [7444B599F3154FAA85AEDFC5A89956A9/DA1FA40C05D541C483DBE4EDA960A628] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-18 00:58:11,124 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:11,124 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533090981_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7444B599F3154FAA85AEDFC5A89956A9/DA1FA40C05D541C483DBE4EDA960A628/.tez/application_1463533090981_0001_wd/localmode-log-dir/dag_1463533090981_0001_1.dot
2016-05-18 00:58:11,125 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7444B599F3154FAA85AEDFC5A89956A9/DA1FA40C05D541C483DBE4EDA960A628/.tez/application_1463533090981_0001_wd/localmode-log-dir/dag_1463533090981_0001_1-tez-dag.pb.txt
2016-05-18 00:58:11,128 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533090981_0001_1
2016-05-18 00:58:11,128 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:11,128 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [7444B599F3154FAA85AEDFC5A89956A9/DA1FA40C05D541C483DBE4EDA960A628] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-18 00:58:11 Running Dag: dag_1463533090981_0001_1
2016-05-18 00:58:11,138 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090981_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533090981_0001_1, submitTime=1463533091123
2016-05-18 00:58:11,141 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7444B599F3154FAA85AEDFC5A89956A9/DA1FA40C05D541C483DBE4EDA960A628/.tez/application_1463533090981_0001/tez-conf.pb" } size: 36913 timestamp: 1463533090000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:11,141 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A]
2016-05-18 00:58:11,141 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A]
2016-05-18 00:58:11,142 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:11,142 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:11,143 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090981_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533090981_0001_1, initTime=1463533091143
2016-05-18 00:58:11,143 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533090981_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:11,144 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:11,144 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090981_0001_1][Event:DAG_STARTED]: dagID=dag_1463533090981_0001_1, startTime=1463533091144
2016-05-18 00:58:11,144 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533090981_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:11,144 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: DD83A54B41CB4663BAB8BBD95E2AAD4A : {0558E434E8D5468B8871B7246DE30CB2={InputName=0558E434E8D5468B8871B7246DE30CB2}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:11,144 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 0558E434E8D5468B8871B7246DE30CB2, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:11,145 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A]
2016-05-18 00:58:11,145 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A]
2016-05-18 00:58:11,145 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A]
2016-05-18 00:58:11,145 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A]
2016-05-18 00:58:11,146 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ield'_RESULTS.0_VALUES_-1, applicationId=application_1463533090981_0001, dagName=[7444B599F3154FAA85AEDFC5A89956A9/DA1FA40C05D541C483DBE4EDA960A628] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-18 00:58:11,146 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:11,148 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 0558E434E8D5468B8871B7246DE30CB2 on vertex vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A]
2016-05-18 00:58:11,152 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 0558E434E8D5468B8871B7246DE30CB2 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:11,154 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:11,154 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:11,154 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:11,154 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:11,155 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 0558E434E8D5468B8871B7246DE30CB2 on vertex vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A]
2016-05-18 00:58:11,156 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:11,157 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {0558E434E8D5468B8871B7246DE30CB2=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:11,157 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A] parallelism set to 1
2016-05-18 00:58:11,157 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:11,158 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A], numAdditionalOutputs=1
2016-05-18 00:58:11,158 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=09785C0FA040423CBE88833101F7E139, vertex=vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:11,160 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533090981_0001, with dag id: dag_1463533090981_0001_1
2016-05-18 00:58:11,160 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for DD83A54B41CB4663BAB8BBD95E2AAD4A:09785C0FA040423CBE88833101F7E139 using oldmapred API
2016-05-18 00:58:11,160 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=09785C0FA040423CBE88833101F7E139, vertexName=DD83A54B41CB4663BAB8BBD95E2AAD4A, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:11,162 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090981_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DD83A54B41CB4663BAB8BBD95E2AAD4A, vertexId=vertex_1463533090981_0001_1_00, initRequestedTime=1463533091144, initedTime=1463533091158, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:11,162 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A] with distanceFromRoot: 0
2016-05-18 00:58:11,162 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:11,163 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090981_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533090981_0001_1_00, startRequestedTime=1463533091146, startedTime=1463533091162
2016-05-18 00:58:11,163 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090981_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533090981_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:11,164 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:11,164 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in DD83A54B41CB4663BAB8BBD95E2AAD4A
2016-05-18 00:58:11,164 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A]
2016-05-18 00:58:11,165 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090981_0001_1][Event:TASK_STARTED]: vertexName=DD83A54B41CB4663BAB8BBD95E2AAD4A, taskId=task_1463533090981_0001_1_00_000000, scheduledTime=1463533091164, launchTime=1463533091164
2016-05-18 00:58:11,165 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A]
2016-05-18 00:58:11,166 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:11,168 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533090981_0001_00_000001, launchTime=1463533091168
2016-05-18 00:58:11,168 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533090981_0001_00_000001
2016-05-18 00:58:11,169 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533090981_0001_1_00_000000_0] started. Is using containerId: [container_1463533090981_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:11,169 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090981_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DD83A54B41CB4663BAB8BBD95E2AAD4A, taskAttemptId=attempt_1463533090981_0001_1_00_000000_0, startTime=1463533091169, containerId=container_1463533090981_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:11,169 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533090981_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533090981_0001_1_00_000000_0
2016-05-18 00:58:11 Starting to run new task attempt: attempt_1463533090981_0001_1_00_000000_0
2016-05-18 00:58:11,170 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533090981_0001_1_00_000000_0
2016-05-18 00:58:11,170 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:11,170 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:11,171 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7444B599F3154FAA85AEDFC5A89956A9/DA1FA40C05D541C483DBE4EDA960A628] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1, VertexName: DD83A54B41CB4663BAB8BBD95E2AAD4A, VertexParallelism: 1, TaskAttemptID:attempt_1463533090981_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0558E434E8D5468B8871B7246DE30CB2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=09785C0FA040423CBE88833101F7E139, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:11,171 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:11,171 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533090981_0001_1_00_000000_0
2016-05-18 00:58:11,172 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533090981_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:11,172 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533090981_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:11,174 INFO input.MRInput (MRInput.java:initialize(440)) - 0558E434E8D5468B8871B7246DE30CB2 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:11,174 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 0558E434E8D5468B8871B7246DE30CB2
2016-05-18 00:58:11,175 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:11,182 INFO output.MROutput (MROutput.java:initialize(421)) - 09785C0FA040423CBE88833101F7E139: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:11,183 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DD83A54B41CB4663BAB8BBD95E2AAD4A, ordinal: 0
2016-05-18 00:58:11,183 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD83A54B41CB4663BAB8BBD95E2AAD4A, mem on start (mb), free: 121, total: 383, max: 672
2016-05-18 00:58:11,183 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:11,183 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:11,183 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:11,183 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:11,184 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[0558E434E8D5468B8871B7246DE30CB2:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [09785C0FA040423CBE88833101F7E139:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:11,184 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:11,184 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:11,184 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:11,185 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[0558E434E8D5468B8871B7246DE30CB2:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [09785C0FA040423CBE88833101F7E139:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:11,185 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:11,185 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 0558E434E8D5468B8871B7246DE30CB2
2016-05-18 00:58:11,185 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 0558E434E8D5468B8871B7246DE30CB2 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:11,186 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:11,186 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:11,186 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:11,187 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533090981_0001_1_00_000000_0
2016-05-18 00:58:11,188 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:11,188 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 0558E434E8D5468B8871B7246DE30CB2 initialized RecordReader from event
2016-05-18 00:58:11,190 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:11,191 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 0558E434E8D5468B8871B7246DE30CB2
2016-05-18 00:58:11,191 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_-1"], id: 09785C0FA040423CBE88833101F7E139
2016-05-18 00:58:11,191 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_-1"]
2016-05-18 00:58:11,192 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:11,192 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DD83A54B41CB4663BAB8BBD95E2AAD4A, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:11,193 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 00:58:11,198 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 00:58:11,204 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 00:58:11,210 INFO output.MROutput (MROutput.java:close(526)) - 09785C0FA040423CBE88833101F7E139 closed
2016-05-18 00:58:11,211 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533090981_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:11,212 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533090981_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:11,212 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533090981_0001_1_00_000000_0: Counters: 8 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463533091192, Process_Duration=20, Process_End_Time=1463533091212, Read_Duration=1, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-18 00:58:11,212 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(817)) - Joining on EventRouter
2016-05-18 00:58:11,213 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD83A54B41CB4663BAB8BBD95E2AAD4A, mem on close (mb), free: 119, total: 383, max: 672
2016-05-18 00:58:11,213 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(851)) - Closed processor for vertex=DD83A54B41CB4663BAB8BBD95E2AAD4A, index=0, interruptedStatus=false
2016-05-18 00:58:11,213 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(887)) - Closed input for vertex=DD83A54B41CB4663BAB8BBD95E2AAD4A, sourceVertex=0558E434E8D5468B8871B7246DE30CB2, interruptedStatus=true
2016-05-18 00:58:11,214 INFO output.MROutput (MROutput.java:close(526)) - 09785C0FA040423CBE88833101F7E139 closed
2016-05-18 00:58:11,214 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(911)) - Closed input for vertex=DD83A54B41CB4663BAB8BBD95E2AAD4A, sourceVertex=09785C0FA040423CBE88833101F7E139, interruptedStatus=true
2016-05-18 00:58:11,216 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533090981_0001_1_00_000000_0 : TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 00:58:11,216 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 00:58:11,216 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533090981_0001_00_000001
2016-05-18 00:58:11,216 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533090981_0001_00_000001
2016-05-18 00:58:11,216 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090981_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DD83A54B41CB4663BAB8BBD95E2AAD4A, taskAttemptId=attempt_1463533090981_0001_1_00_000000_0, creationTime=1463533091165, allocationTime=1463533091167, startTime=1463533091169, finishTime=1463533091216, timeTaken=47, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task: attempt_1463533090981_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
, counters=Counters: 9, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463533091192, Process_Duration=20, Process_End_Time=1463533091212, Read_Duration=1, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-18 00:58:11,216 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533090981_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, errorMessage='TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation'}
2016-05-18 00:58:11,216 INFO impl.TaskImpl (TaskImpl.java:transition(1170)) - Failing task: task_1463533090981_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-18 00:58:11,217 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090981_0001_1][Event:TASK_FINISHED]: vertexName=DD83A54B41CB4663BAB8BBD95E2AAD4A, taskId=task_1463533090981_0001_1_00_000000, startTime=1463533091169, finishTime=1463533091217, timeTaken=48, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463533090981_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
], counters=Counters: 0
2016-05-18 00:58:11,217 INFO impl.VertexImpl (VertexImpl.java:transition(3353)) - Failing vertex: vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A] because task failed: task_1463533090981_0001_1_00_000000
2016-05-18 00:58:11,218 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(2102)) - Killing tasks in vertex: vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A] due to trigger: OWN_TASK_FAILURE
2016-05-18 00:58:11,218 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A], tasks=1, failed=1, killed=0, success=0, completed=1, commits=0, err=OWN_TASK_FAILURE
2016-05-18 00:58:11,218 INFO impl.VertexImpl (VertexImpl.java:finishWithTerminationCause(2084)) - Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
2016-05-18 00:58:11,218 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3087)) - Invoking committer abort for vertex, vertexId=vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A]
2016-05-18 00:58:11,218 INFO impl.VertexImpl (VertexImpl.java:run(3095)) - Invoking committer abort for output=09785C0FA040423CBE88833101F7E139, vertexId=vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A]
2016-05-18 00:58:11,220 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090981_0001_1][Event:VERTEX_FINISHED]: vertexName=DD83A54B41CB4663BAB8BBD95E2AAD4A, vertexId=vertex_1463533090981_0001_1_00, initRequestedTime=1463533091144, initedTime=1463533091158, startRequestedTime=1463533091146, startedTime=1463533091162, finishTime=1463533091219, timeTaken=57, status=FAILED, diagnostics=Task failed, taskId=task_1463533090981_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463533090981_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]]
Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
Vertex vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A] killed/failed due to:OWN_TASK_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=1463533091169, firstTasksToStart=[ task_1463533090981_0001_1_00_000000 ], lastTaskFinishTime=1463533091216, lastTasksToFinish=[ task_1463533090981_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-18 00:58:11,221 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-18 00:58:11,221 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-18 00:58:11,221 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=VERTEX_FAILURE
2016-05-18 00:58:11,221 INFO impl.DAGImpl (DAGImpl.java:finishWithTerminationCause(1350)) - DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
2016-05-18 00:58:11,222 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(294)) - Container: container_1463533090981_0001_00_000001 completed but with errors
2016-05-18 00:58:11,224 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533090981_0001_1, queueSize=0
2016-05-18 00:58:11,224 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090981_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533090981_0001_1, startTime=1463533091144, finishTime=1463533091222, timeTaken=78, status=FAILED, diagnostics=Vertex failed, vertexName=DD83A54B41CB4663BAB8BBD95E2AAD4A, vertexId=vertex_1463533090981_0001_1_00, diagnostics=[Task failed, taskId=task_1463533090981_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463533090981_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A] killed/failed due to:OWN_TASK_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0, counters=Counters: 4, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, AM_CPU_MILLISECONDS=150, AM_GC_TIME_MILLIS=0
2016-05-18 00:58:11,224 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3082)) - Ignoring multiple aborts for vertex: vertex_1463533090981_0001_1_00 [DD83A54B41CB4663BAB8BBD95E2AAD4A]
2016-05-18 00:58:11,225 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533090981_0001_1 finished with state: FAILED
2016-05-18 00:58:11,225 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533090981_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:11,226 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533090981_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533090981_0001_00_000001, stoppedTime=1463533091226, exitStatus=-1000
2016-05-18 00:58:11,226 INFO node.AMNodeImpl (AMNodeImpl.java:transition(298)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463533090981_0001_1_00_000000_0 failed: true container: container_1463533090981_0001_00_000001 numFailedTAs: 0
2016-05-18 00:58:11,229 INFO container.AMContainerImpl (AMContainerImpl.java:transition(993)) - Container container_1463533090981_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
2016-05-18 00:58:11 Completed Dag: dag_1463533090981_0001_1
2016-05-18 00:58:11,232 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533090981_0001_1, dagState=FAILED
2016-05-18 00:58:11,232 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533090981_0001_1_post
2016-05-18 00:58:11,232 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:11,234 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 3
2016-05-18 00:58:11,234 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:11,234 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(260)) - Ignoring stop request for containerId: container_1463533090981_0001_00_000001
2016-05-18 00:58:11,234 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[7444B599F3154FAA85AEDFC5A89956A9/DA1FA40C05D541C483DBE4EDA960A628] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1, with id=dag_1463533090981_0001_1
2016-05-18 00:58:11,235 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[7444B599F3154FAA85AEDFC5A89956A9/DA1FA40C05D541C483DBE4EDA960A628] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1, with id=dag_1463533090981_0001_1
2016-05-18 00:58:11,236 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ield'_RESULTS.0_VALUES_-1, applicationId=application_1463533090981_0001
2016-05-18 00:58:11,238 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:11,238 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:11,238 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:11,238 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:11,238 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:11,239 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:11,239 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:11,239 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:11,239 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:11,240 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:11,240 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:11,241 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:11,241 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:11,242 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:11,245 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7444B599F3154FAA85AEDFC5A89956A9/DA1FA40C05D541C483DBE4EDA960A628/.tez/application_1463533090981_0001
2016-05-18 00:58:11,245 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-18 00:58:11,246 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-18 00:58:11,246 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-18 00:58:11,247 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_-1/_temporary
2016-05-18 00:58:11,248 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.285
2016-05-18 00:58:11,248 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_VALUES_-1
2016-05-18 00:58:11,285 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:11,286 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:11,314 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-18 00:58:11,317 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.006
2016-05-18 00:58:11,318 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 00:58:11,318 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 00:58:11,318 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_VALUES_'field2'
2016-05-18 00:58:11,346 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:11,347 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:11,388 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-18 00:58:11,396 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-18 00:58:11,397 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:11,397 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:11,397 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:11,412 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:11,412 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_RESULTS
2016-05-18 00:58:11,423 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:11,424 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:11,424 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_RESULTS"]
2016-05-18 00:58:11,424 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:11,425 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:11,425 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:11,427 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-18 00:58:11,427 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:11,443 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:11,448 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:11,448 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:11,448 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:11,458 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/42D0B9399311492BA19D4AF758137B33/AAC18E00C22D4F6B87398526721E3EC6/.tez/application_1463533091448_0001 doesn't exist and is created
2016-05-18 00:58:11,473 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:11,473 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:11,474 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/42D0B9399311492BA19D4AF758137B33/AAC18E00C22D4F6B87398526721E3EC6/.tez/application_1463533091448_0001_wd
2016-05-18 00:58:11,484 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533091448_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:11,485 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:11,485 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:11,486 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:11,487 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:11,487 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:11,487 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:11,488 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:11,490 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:11,490 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:11,490 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:11,490 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:11,490 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:11,491 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:11,491 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:11,491 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:11,491 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:11,491 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:11,491 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:11,491 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:11,494 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:11,495 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:11,495 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:11,496 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/42D0B9399311492BA19D4AF758137B33/AAC18E00C22D4F6B87398526721E3EC6/.tez/application_1463533091448_0001_wd/localmode-log-dir
2016-05-18 00:58:11,496 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/42D0B9399311492BA19D4AF758137B33/AAC18E00C22D4F6B87398526721E3EC6/.tez/application_1463533091448_0001_wd/localmode-log-dir/history.txt.appattempt_1463533091448_0001_000000, maxErrors=10
2016-05-18 00:58:11,496 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/42D0B9399311492BA19D4AF758137B33/AAC18E00C22D4F6B87398526721E3EC6/.tez/application_1463533091448_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:11,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533091448_0001_000000, appSubmitTime=1463533091484, launchTime=1463533091484
2016-05-18 00:58:11,503 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:60458
2016-05-18 00:58:11,505 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533091448
2016-05-18 00:58:11,506 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:11,510 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:58:11,510 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:58:11,512 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533091448_0001_000000, startTime=1463533091512
2016-05-18 00:58:11,512 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:11,574 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:11,574 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:11,574 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ..._RESULTS.0_VALUES_RESULTS, applicationId=application_1463533091448_0001, dagName=[42D0B9399311492BA19D4AF758137B33/AAC18E00C22D4F6B87398526721E3EC6] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-18 00:58:11,591 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [42D0B9399311492BA19D4AF758137B33/AAC18E00C22D4F6B87398526721E3EC6] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-18 00:58:11,592 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:11,593 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533091448_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/42D0B9399311492BA19D4AF758137B33/AAC18E00C22D4F6B87398526721E3EC6/.tez/application_1463533091448_0001_wd/localmode-log-dir/dag_1463533091448_0001_1.dot
2016-05-18 00:58:11,593 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/42D0B9399311492BA19D4AF758137B33/AAC18E00C22D4F6B87398526721E3EC6/.tez/application_1463533091448_0001_wd/localmode-log-dir/dag_1463533091448_0001_1-tez-dag.pb.txt
2016-05-18 00:58:11,597 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533091448_0001_1
2016-05-18 00:58:11,597 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:11,597 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [42D0B9399311492BA19D4AF758137B33/AAC18E00C22D4F6B87398526721E3EC6] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-18 00:58:11 Running Dag: dag_1463533091448_0001_1
2016-05-18 00:58:11,607 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091448_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533091448_0001_1, submitTime=1463533091592
2016-05-18 00:58:11,608 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/42D0B9399311492BA19D4AF758137B33/AAC18E00C22D4F6B87398526721E3EC6/.tez/application_1463533091448_0001/tez-conf.pb" } size: 36913 timestamp: 1463533091000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:11,608 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B]
2016-05-18 00:58:11,611 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B]
2016-05-18 00:58:11,611 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:11,611 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:11,613 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091448_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533091448_0001_1, initTime=1463533091613
2016-05-18 00:58:11,613 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533091448_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:11,613 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:11,614 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091448_0001_1][Event:DAG_STARTED]: dagID=dag_1463533091448_0001_1, startTime=1463533091614
2016-05-18 00:58:11,614 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533091448_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:11,614 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: A2D807F421BB4BC8871ED531F555762B : {C62FBA4F5C8A4D3A8FA376F92282A53D={InputName=C62FBA4F5C8A4D3A8FA376F92282A53D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:11,614 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: C62FBA4F5C8A4D3A8FA376F92282A53D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:11,614 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B]
2016-05-18 00:58:11,614 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B]
2016-05-18 00:58:11,615 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B]
2016-05-18 00:58:11,615 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B]
2016-05-18 00:58:11,615 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:11,616 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: C62FBA4F5C8A4D3A8FA376F92282A53D on vertex vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B]
2016-05-18 00:58:11,616 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ..._RESULTS.0_VALUES_RESULTS, applicationId=application_1463533091448_0001, dagName=[42D0B9399311492BA19D4AF758137B33/AAC18E00C22D4F6B87398526721E3EC6] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-18 00:58:11,618 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input C62FBA4F5C8A4D3A8FA376F92282A53D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:11,620 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:11,620 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:11,620 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:11,620 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:11,621 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: C62FBA4F5C8A4D3A8FA376F92282A53D on vertex vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B]
2016-05-18 00:58:11,622 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:11,622 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {C62FBA4F5C8A4D3A8FA376F92282A53D=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:11,623 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B] parallelism set to 1
2016-05-18 00:58:11,623 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:11,624 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B], numAdditionalOutputs=1
2016-05-18 00:58:11,624 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=6AF2CC3B8F8E40CEA7FE250DB946A34C, vertex=vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:11,626 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for A2D807F421BB4BC8871ED531F555762B:6AF2CC3B8F8E40CEA7FE250DB946A34C using oldmapred API
2016-05-18 00:58:11,626 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=6AF2CC3B8F8E40CEA7FE250DB946A34C, vertexName=A2D807F421BB4BC8871ED531F555762B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:11,627 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091448_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A2D807F421BB4BC8871ED531F555762B, vertexId=vertex_1463533091448_0001_1_00, initRequestedTime=1463533091614, initedTime=1463533091624, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:11,627 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B] with distanceFromRoot: 0
2016-05-18 00:58:11,627 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:11,628 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091448_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533091448_0001_1_00, startRequestedTime=1463533091616, startedTime=1463533091628
2016-05-18 00:58:11,629 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091448_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533091448_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:11,629 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:11,629 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in A2D807F421BB4BC8871ED531F555762B
2016-05-18 00:58:11,629 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B]
2016-05-18 00:58:11,630 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091448_0001_1][Event:TASK_STARTED]: vertexName=A2D807F421BB4BC8871ED531F555762B, taskId=task_1463533091448_0001_1_00_000000, scheduledTime=1463533091630, launchTime=1463533091630
2016-05-18 00:58:11,630 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B]
2016-05-18 00:58:11,631 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:11,634 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533091448_0001_00_000001, launchTime=1463533091634
2016-05-18 00:58:11,635 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533091448_0001_00_000001
2016-05-18 00:58:11,638 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533091448_0001_1_00_000000_0] started. Is using containerId: [container_1463533091448_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:11,638 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091448_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A2D807F421BB4BC8871ED531F555762B, taskAttemptId=attempt_1463533091448_0001_1_00_000000_0, startTime=1463533091637, containerId=container_1463533091448_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:11,639 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533091448_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533091448_0001_1_00_000000_0
2016-05-18 00:58:11 Starting to run new task attempt: attempt_1463533091448_0001_1_00_000000_0
2016-05-18 00:58:11,639 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533091448_0001_1_00_000000_0
2016-05-18 00:58:11,640 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:11,641 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:11,641 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [42D0B9399311492BA19D4AF758137B33/AAC18E00C22D4F6B87398526721E3EC6] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS, VertexName: A2D807F421BB4BC8871ED531F555762B, VertexParallelism: 1, TaskAttemptID:attempt_1463533091448_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C62FBA4F5C8A4D3A8FA376F92282A53D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=6AF2CC3B8F8E40CEA7FE250DB946A34C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:11,641 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:11,642 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533091448_0001_1_00_000000_0
2016-05-18 00:58:11,643 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533091448_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:11,643 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533091448_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:11,644 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533091448_0001, with dag id: dag_1463533091448_0001_1
2016-05-18 00:58:11,647 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:11,654 INFO input.MRInput (MRInput.java:initialize(440)) - C62FBA4F5C8A4D3A8FA376F92282A53D using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:11,654 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: C62FBA4F5C8A4D3A8FA376F92282A53D
2016-05-18 00:58:11,655 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A2D807F421BB4BC8871ED531F555762B, ordinal: 0
2016-05-18 00:58:11,655 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A2D807F421BB4BC8871ED531F555762B, mem on start (mb), free: 221, total: 368, max: 672
2016-05-18 00:58:11,655 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:11,655 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:11,655 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:11,658 INFO output.MROutput (MROutput.java:initialize(421)) - 6AF2CC3B8F8E40CEA7FE250DB946A34C: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:11,658 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:11,658 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[C62FBA4F5C8A4D3A8FA376F92282A53D:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [6AF2CC3B8F8E40CEA7FE250DB946A34C:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:11,659 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:11,659 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:11,659 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:11,660 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[C62FBA4F5C8A4D3A8FA376F92282A53D:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [6AF2CC3B8F8E40CEA7FE250DB946A34C:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:11,660 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:11,660 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: C62FBA4F5C8A4D3A8FA376F92282A53D
2016-05-18 00:58:11,660 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: C62FBA4F5C8A4D3A8FA376F92282A53D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:11,661 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:11,661 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:11,661 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:11,662 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533091448_0001_1_00_000000_0
2016-05-18 00:58:11,665 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:11,666 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: C62FBA4F5C8A4D3A8FA376F92282A53D
2016-05-18 00:58:11,666 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_RESULTS"], id: 6AF2CC3B8F8E40CEA7FE250DB946A34C
2016-05-18 00:58:11,667 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:11,667 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - C62FBA4F5C8A4D3A8FA376F92282A53D initialized RecordReader from event
2016-05-18 00:58:11,667 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_RESULTS"]
2016-05-18 00:58:11,667 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:11,668 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A2D807F421BB4BC8871ED531F555762B, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:11,668 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 00:58:11,669 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 00:58:11,670 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 00:58:11,671 INFO output.MROutput (MROutput.java:close(526)) - 6AF2CC3B8F8E40CEA7FE250DB946A34C closed
2016-05-18 00:58:11,671 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533091448_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:11,672 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533091448_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:11,672 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533091448_0001_1_00_000000_0: Counters: 8 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463533091668, Process_Duration=4, Process_End_Time=1463533091672, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-18 00:58:11,673 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(817)) - Joining on EventRouter
2016-05-18 00:58:11,673 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A2D807F421BB4BC8871ED531F555762B, mem on close (mb), free: 219, total: 368, max: 672
2016-05-18 00:58:11,673 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(851)) - Closed processor for vertex=A2D807F421BB4BC8871ED531F555762B, index=0, interruptedStatus=false
2016-05-18 00:58:11,673 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(887)) - Closed input for vertex=A2D807F421BB4BC8871ED531F555762B, sourceVertex=C62FBA4F5C8A4D3A8FA376F92282A53D, interruptedStatus=true
2016-05-18 00:58:11,673 INFO output.MROutput (MROutput.java:close(526)) - 6AF2CC3B8F8E40CEA7FE250DB946A34C closed
2016-05-18 00:58:11,674 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(911)) - Closed input for vertex=A2D807F421BB4BC8871ED531F555762B, sourceVertex=6AF2CC3B8F8E40CEA7FE250DB946A34C, interruptedStatus=true
2016-05-18 00:58:11,675 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533091448_0001_1_00_000000_0 : TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 00:58:11,675 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 00:58:11,675 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091448_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A2D807F421BB4BC8871ED531F555762B, taskAttemptId=attempt_1463533091448_0001_1_00_000000_0, creationTime=1463533091630, allocationTime=1463533091633, startTime=1463533091637, finishTime=1463533091675, timeTaken=38, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task: attempt_1463533091448_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
, counters=Counters: 9, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463533091668, Process_Duration=4, Process_End_Time=1463533091672, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-18 00:58:11,677 INFO impl.TaskImpl (TaskImpl.java:transition(1170)) - Failing task: task_1463533091448_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-18 00:58:11,677 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091448_0001_1][Event:TASK_FINISHED]: vertexName=A2D807F421BB4BC8871ED531F555762B, taskId=task_1463533091448_0001_1_00_000000, startTime=1463533091637, finishTime=1463533091677, timeTaken=40, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463533091448_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
], counters=Counters: 0
2016-05-18 00:58:11,678 INFO impl.VertexImpl (VertexImpl.java:transition(3353)) - Failing vertex: vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B] because task failed: task_1463533091448_0001_1_00_000000
2016-05-18 00:58:11,678 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(2102)) - Killing tasks in vertex: vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B] due to trigger: OWN_TASK_FAILURE
2016-05-18 00:58:11,678 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B], tasks=1, failed=1, killed=0, success=0, completed=1, commits=0, err=OWN_TASK_FAILURE
2016-05-18 00:58:11,678 INFO impl.VertexImpl (VertexImpl.java:finishWithTerminationCause(2084)) - Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
2016-05-18 00:58:11,678 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3087)) - Invoking committer abort for vertex, vertexId=vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B]
2016-05-18 00:58:11,678 INFO impl.VertexImpl (VertexImpl.java:run(3095)) - Invoking committer abort for output=6AF2CC3B8F8E40CEA7FE250DB946A34C, vertexId=vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B]
2016-05-18 00:58:11,675 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533091448_0001_00_000001
2016-05-18 00:58:11,679 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533091448_0001_00_000001
2016-05-18 00:58:11,680 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533091448_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, errorMessage='TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation'}
2016-05-18 00:58:11,680 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091448_0001_1][Event:VERTEX_FINISHED]: vertexName=A2D807F421BB4BC8871ED531F555762B, vertexId=vertex_1463533091448_0001_1_00, initRequestedTime=1463533091614, initedTime=1463533091624, startRequestedTime=1463533091616, startedTime=1463533091628, finishTime=1463533091680, timeTaken=52, status=FAILED, diagnostics=Task failed, taskId=task_1463533091448_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463533091448_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]]
Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
Vertex vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B] killed/failed due to:OWN_TASK_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=1463533091637, firstTasksToStart=[ task_1463533091448_0001_1_00_000000 ], lastTaskFinishTime=1463533091675, lastTasksToFinish=[ task_1463533091448_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-18 00:58:11,681 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-18 00:58:11,680 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(294)) - Container: container_1463533091448_0001_00_000001 completed but with errors
2016-05-18 00:58:11,681 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091448_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533091448_0001_00_000001, stoppedTime=1463533091681, exitStatus=-1000
2016-05-18 00:58:11,681 INFO node.AMNodeImpl (AMNodeImpl.java:transition(298)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463533091448_0001_1_00_000000_0 failed: true container: container_1463533091448_0001_00_000001 numFailedTAs: 0
2016-05-18 00:58:11,682 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-18 00:58:11,682 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=VERTEX_FAILURE
2016-05-18 00:58:11,682 INFO impl.DAGImpl (DAGImpl.java:finishWithTerminationCause(1350)) - DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
2016-05-18 00:58:11,684 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533091448_0001_1, queueSize=0
2016-05-18 00:58:11,684 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091448_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533091448_0001_1, startTime=1463533091613, finishTime=1463533091683, timeTaken=70, status=FAILED, diagnostics=Vertex failed, vertexName=A2D807F421BB4BC8871ED531F555762B, vertexId=vertex_1463533091448_0001_1_00, diagnostics=[Task failed, taskId=task_1463533091448_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463533091448_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B] killed/failed due to:OWN_TASK_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0, counters=Counters: 4, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, AM_CPU_MILLISECONDS=200, AM_GC_TIME_MILLIS=0
2016-05-18 00:58:11,685 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3082)) - Ignoring multiple aborts for vertex: vertex_1463533091448_0001_1_00 [A2D807F421BB4BC8871ED531F555762B]
2016-05-18 00:58:11,685 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533091448_0001_1 finished with state: FAILED
2016-05-18 00:58:11,685 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533091448_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:11,685 INFO container.AMContainerImpl (AMContainerImpl.java:transition(993)) - Container container_1463533091448_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
2016-05-18 00:58:11,686 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(260)) - Ignoring stop request for containerId: container_1463533091448_0001_00_000001
2016-05-18 00:58:11 Completed Dag: dag_1463533091448_0001_1
2016-05-18 00:58:11,686 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533091448_0001_1, dagState=FAILED
2016-05-18 00:58:11,686 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533091448_0001_1_post
2016-05-18 00:58:11,686 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:11,686 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 2
2016-05-18 00:58:11,687 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:11,687 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[42D0B9399311492BA19D4AF758137B33/AAC18E00C22D4F6B87398526721E3EC6] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS, with id=dag_1463533091448_0001_1
2016-05-18 00:58:11,687 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[42D0B9399311492BA19D4AF758137B33/AAC18E00C22D4F6B87398526721E3EC6] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS, with id=dag_1463533091448_0001_1
2016-05-18 00:58:11,697 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ..._RESULTS.0_VALUES_RESULTS, applicationId=application_1463533091448_0001
2016-05-18 00:58:11,699 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:11,699 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:11,699 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:11,699 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:11,699 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:11,700 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:11,700 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:11,700 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:11,701 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:11,701 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:11,701 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:11,701 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:11,701 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:11,702 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:11,707 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/42D0B9399311492BA19D4AF758137B33/AAC18E00C22D4F6B87398526721E3EC6/.tez/application_1463533091448_0001
2016-05-18 00:58:11,707 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-18 00:58:11,707 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-18 00:58:11,708 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-18 00:58:11,708 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_RESULTS/_temporary
2016-05-18 00:58:11,709 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.285
2016-05-18 00:58:11,709 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_VALUES_RESULTS
2016-05-18 00:58:11,733 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:11,733 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:11,769 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-18 00:58:11,775 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-18 00:58:11,776 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:11,776 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:11,776 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:11,791 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:11,791 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_ALL
2016-05-18 00:58:11,800 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:11,800 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:11,801 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_ALL"]
2016-05-18 00:58:11,801 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:11,801 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:11,801 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:11,802 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-18 00:58:11,802 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:11,815 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:11,820 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:11,820 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:11,820 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:11,829 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FCA9FC9C950A4A5A95B9F58419CF7005/CE3EB7B5890742929ECDCB02297FFD3F/.tez/application_1463533091820_0001 doesn't exist and is created
2016-05-18 00:58:11,843 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:11,843 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:11,844 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FCA9FC9C950A4A5A95B9F58419CF7005/CE3EB7B5890742929ECDCB02297FFD3F/.tez/application_1463533091820_0001_wd
2016-05-18 00:58:11,854 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533091820_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:11,854 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:11,854 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:11,856 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:11,856 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:11,856 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:11,856 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:11,856 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:11,858 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:11,858 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:11,859 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:11,859 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:11,859 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:11,859 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:11,859 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:11,859 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:11,859 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:11,859 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:11,860 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:11,860 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:11,861 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:11,862 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:11,862 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:11,862 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FCA9FC9C950A4A5A95B9F58419CF7005/CE3EB7B5890742929ECDCB02297FFD3F/.tez/application_1463533091820_0001_wd/localmode-log-dir
2016-05-18 00:58:11,862 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FCA9FC9C950A4A5A95B9F58419CF7005/CE3EB7B5890742929ECDCB02297FFD3F/.tez/application_1463533091820_0001_wd/localmode-log-dir/history.txt.appattempt_1463533091820_0001_000000, maxErrors=10
2016-05-18 00:58:11,863 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FCA9FC9C950A4A5A95B9F58419CF7005/CE3EB7B5890742929ECDCB02297FFD3F/.tez/application_1463533091820_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:11,863 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533091820_0001_000000, appSubmitTime=1463533091853, launchTime=1463533091853
2016-05-18 00:58:11,865 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:11,869 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:36058
2016-05-18 00:58:11,873 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533091820
2016-05-18 00:58:11,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533091820_0001_000000, startTime=1463533091874
2016-05-18 00:58:11,875 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:11,907 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:58:11,907 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:58:11,944 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:11,944 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:11,944 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.0_VALUES_ALL, applicationId=application_1463533091820_0001, dagName=[FCA9FC9C950A4A5A95B9F58419CF7005/CE3EB7B5890742929ECDCB02297FFD3F] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-18 00:58:11,960 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [FCA9FC9C950A4A5A95B9F58419CF7005/CE3EB7B5890742929ECDCB02297FFD3F] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-18 00:58:11,960 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:11,961 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533091820_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FCA9FC9C950A4A5A95B9F58419CF7005/CE3EB7B5890742929ECDCB02297FFD3F/.tez/application_1463533091820_0001_wd/localmode-log-dir/dag_1463533091820_0001_1.dot
2016-05-18 00:58:11,961 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FCA9FC9C950A4A5A95B9F58419CF7005/CE3EB7B5890742929ECDCB02297FFD3F/.tez/application_1463533091820_0001_wd/localmode-log-dir/dag_1463533091820_0001_1-tez-dag.pb.txt
2016-05-18 00:58:11,965 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533091820_0001_1
2016-05-18 00:58:11,965 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:11,965 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [FCA9FC9C950A4A5A95B9F58419CF7005/CE3EB7B5890742929ECDCB02297FFD3F] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-18 00:58:11 Running Dag: dag_1463533091820_0001_1
2016-05-18 00:58:11,974 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091820_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533091820_0001_1, submitTime=1463533091960
2016-05-18 00:58:11,975 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FCA9FC9C950A4A5A95B9F58419CF7005/CE3EB7B5890742929ECDCB02297FFD3F/.tez/application_1463533091820_0001/tez-conf.pb" } size: 36913 timestamp: 1463533091000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:11,976 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA]
2016-05-18 00:58:11,976 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA]
2016-05-18 00:58:11,977 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:11,977 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:11,978 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091820_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533091820_0001_1, initTime=1463533091978
2016-05-18 00:58:11,978 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533091820_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:11,978 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:11,979 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091820_0001_1][Event:DAG_STARTED]: dagID=dag_1463533091820_0001_1, startTime=1463533091979
2016-05-18 00:58:11,979 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533091820_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:11,979 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: D91C4B2E0DA44E8691A9A49137BDF7BA : {B971B757363E4254B1B541FAF0320673={InputName=B971B757363E4254B1B541FAF0320673}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:11,979 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: B971B757363E4254B1B541FAF0320673, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:11,979 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA]
2016-05-18 00:58:11,980 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA]
2016-05-18 00:58:11,980 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA]
2016-05-18 00:58:11,980 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA]
2016-05-18 00:58:11,980 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:11,984 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: B971B757363E4254B1B541FAF0320673 on vertex vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA]
2016-05-18 00:58:11,984 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.0_VALUES_ALL, applicationId=application_1463533091820_0001, dagName=[FCA9FC9C950A4A5A95B9F58419CF7005/CE3EB7B5890742929ECDCB02297FFD3F] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-18 00:58:11,985 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input B971B757363E4254B1B541FAF0320673 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:11,986 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:11,987 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:11,987 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:11,987 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:11,987 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: B971B757363E4254B1B541FAF0320673 on vertex vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA]
2016-05-18 00:58:11,988 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:11,989 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {B971B757363E4254B1B541FAF0320673=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:11,992 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA] parallelism set to 1
2016-05-18 00:58:11,992 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:11,993 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA], numAdditionalOutputs=1
2016-05-18 00:58:11,993 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=7A8A66F92E7B43CB867B074C6EF5098C, vertex=vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:11,994 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D91C4B2E0DA44E8691A9A49137BDF7BA:7A8A66F92E7B43CB867B074C6EF5098C using oldmapred API
2016-05-18 00:58:11,994 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=7A8A66F92E7B43CB867B074C6EF5098C, vertexName=D91C4B2E0DA44E8691A9A49137BDF7BA, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:11,995 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091820_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D91C4B2E0DA44E8691A9A49137BDF7BA, vertexId=vertex_1463533091820_0001_1_00, initRequestedTime=1463533091979, initedTime=1463533091993, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:11,995 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA] with distanceFromRoot: 0
2016-05-18 00:58:11,996 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:11,996 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091820_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533091820_0001_1_00, startRequestedTime=1463533091981, startedTime=1463533091996
2016-05-18 00:58:11,996 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091820_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533091820_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:11,996 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:11,996 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in D91C4B2E0DA44E8691A9A49137BDF7BA
2016-05-18 00:58:11,997 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA]
2016-05-18 00:58:11,997 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091820_0001_1][Event:TASK_STARTED]: vertexName=D91C4B2E0DA44E8691A9A49137BDF7BA, taskId=task_1463533091820_0001_1_00_000000, scheduledTime=1463533091997, launchTime=1463533091997
2016-05-18 00:58:11,998 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA]
2016-05-18 00:58:11,999 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:12,001 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533091820_0001_00_000001, launchTime=1463533092000
2016-05-18 00:58:12,001 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533091820_0001_00_000001
2016-05-18 00:58:12,002 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533091820_0001_1_00_000000_0] started. Is using containerId: [container_1463533091820_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:12,003 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091820_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D91C4B2E0DA44E8691A9A49137BDF7BA, taskAttemptId=attempt_1463533091820_0001_1_00_000000_0, startTime=1463533092002, containerId=container_1463533091820_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:12,003 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533091820_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533091820_0001_1_00_000000_0
2016-05-18 00:58:12 Starting to run new task attempt: attempt_1463533091820_0001_1_00_000000_0
2016-05-18 00:58:12,004 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533091820_0001_1_00_000000_0
2016-05-18 00:58:12,004 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:12,004 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:12,005 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FCA9FC9C950A4A5A95B9F58419CF7005/CE3EB7B5890742929ECDCB02297FFD3F] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL, VertexName: D91C4B2E0DA44E8691A9A49137BDF7BA, VertexParallelism: 1, TaskAttemptID:attempt_1463533091820_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B971B757363E4254B1B541FAF0320673, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=7A8A66F92E7B43CB867B074C6EF5098C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:12,005 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:12,006 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533091820_0001_1_00_000000_0
2016-05-18 00:58:12,007 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533091820_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:12,007 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533091820_0001, with dag id: dag_1463533091820_0001_1
2016-05-18 00:58:12,007 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533091820_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:12,011 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:12,016 INFO input.MRInput (MRInput.java:initialize(440)) - B971B757363E4254B1B541FAF0320673 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:12,017 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: B971B757363E4254B1B541FAF0320673
2016-05-18 00:58:12,023 INFO output.MROutput (MROutput.java:initialize(421)) - 7A8A66F92E7B43CB867B074C6EF5098C: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:12,023 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D91C4B2E0DA44E8691A9A49137BDF7BA, ordinal: 0
2016-05-18 00:58:12,023 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D91C4B2E0DA44E8691A9A49137BDF7BA, mem on start (mb), free: 152, total: 368, max: 672
2016-05-18 00:58:12,023 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:12,024 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:12,024 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:12,024 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:12,025 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[7A8A66F92E7B43CB867B074C6EF5098C:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [B971B757363E4254B1B541FAF0320673:INPUT:0:org.apache.tez.mapreduce.input.MRInput]
2016-05-18 00:58:12,025 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:12,026 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:12,026 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:12,026 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[7A8A66F92E7B43CB867B074C6EF5098C:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [B971B757363E4254B1B541FAF0320673:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0]
2016-05-18 00:58:12,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:12,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: B971B757363E4254B1B541FAF0320673 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:12,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: B971B757363E4254B1B541FAF0320673
2016-05-18 00:58:12,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:12,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:12,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:12,029 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533091820_0001_1_00_000000_0
2016-05-18 00:58:12,030 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:12,031 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - B971B757363E4254B1B541FAF0320673 initialized RecordReader from event
2016-05-18 00:58:12,033 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:12,033 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: B971B757363E4254B1B541FAF0320673
2016-05-18 00:58:12,034 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_ALL"], id: 7A8A66F92E7B43CB867B074C6EF5098C
2016-05-18 00:58:12,034 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_ALL"]
2016-05-18 00:58:12,034 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:12,035 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D91C4B2E0DA44E8691A9A49137BDF7BA, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:12,036 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 00:58:12,036 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 00:58:12,037 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 00:58:12,038 INFO output.MROutput (MROutput.java:close(526)) - 7A8A66F92E7B43CB867B074C6EF5098C closed
2016-05-18 00:58:12,038 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533091820_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:12,040 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533091820_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:12,040 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533091820_0001_1_00_000000_0: Counters: 8 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463533092035, Process_Duration=5, Process_End_Time=1463533092040, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-18 00:58:12,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(817)) - Joining on EventRouter
2016-05-18 00:58:12,041 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D91C4B2E0DA44E8691A9A49137BDF7BA, mem on close (mb), free: 150, total: 368, max: 672
2016-05-18 00:58:12,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(851)) - Closed processor for vertex=D91C4B2E0DA44E8691A9A49137BDF7BA, index=0, interruptedStatus=false
2016-05-18 00:58:12,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(887)) - Closed input for vertex=D91C4B2E0DA44E8691A9A49137BDF7BA, sourceVertex=B971B757363E4254B1B541FAF0320673, interruptedStatus=true
2016-05-18 00:58:12,041 INFO output.MROutput (MROutput.java:close(526)) - 7A8A66F92E7B43CB867B074C6EF5098C closed
2016-05-18 00:58:12,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(911)) - Closed input for vertex=D91C4B2E0DA44E8691A9A49137BDF7BA, sourceVertex=7A8A66F92E7B43CB867B074C6EF5098C, interruptedStatus=true
2016-05-18 00:58:12,042 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533091820_0001_1_00_000000_0 : TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 00:58:12,043 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 00:58:12,043 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533091820_0001_00_000001
2016-05-18 00:58:12,043 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533091820_0001_00_000001
2016-05-18 00:58:12,043 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533091820_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, errorMessage='TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation'}
2016-05-18 00:58:12,043 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091820_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D91C4B2E0DA44E8691A9A49137BDF7BA, taskAttemptId=attempt_1463533091820_0001_1_00_000000_0, creationTime=1463533091998, allocationTime=1463533092000, startTime=1463533092002, finishTime=1463533092043, timeTaken=41, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task: attempt_1463533091820_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
, counters=Counters: 9, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463533092035, Process_Duration=5, Process_End_Time=1463533092040, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-18 00:58:12,044 INFO impl.TaskImpl (TaskImpl.java:transition(1170)) - Failing task: task_1463533091820_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-18 00:58:12,044 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091820_0001_1][Event:TASK_FINISHED]: vertexName=D91C4B2E0DA44E8691A9A49137BDF7BA, taskId=task_1463533091820_0001_1_00_000000, startTime=1463533092002, finishTime=1463533092044, timeTaken=42, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463533091820_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
], counters=Counters: 0
2016-05-18 00:58:12,044 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(294)) - Container: container_1463533091820_0001_00_000001 completed but with errors
2016-05-18 00:58:12,045 INFO impl.VertexImpl (VertexImpl.java:transition(3353)) - Failing vertex: vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA] because task failed: task_1463533091820_0001_1_00_000000
2016-05-18 00:58:12,045 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(2102)) - Killing tasks in vertex: vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA] due to trigger: OWN_TASK_FAILURE
2016-05-18 00:58:12,045 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA], tasks=1, failed=1, killed=0, success=0, completed=1, commits=0, err=OWN_TASK_FAILURE
2016-05-18 00:58:12,045 INFO impl.VertexImpl (VertexImpl.java:finishWithTerminationCause(2084)) - Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
2016-05-18 00:58:12,045 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3087)) - Invoking committer abort for vertex, vertexId=vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA]
2016-05-18 00:58:12,046 INFO impl.VertexImpl (VertexImpl.java:run(3095)) - Invoking committer abort for output=7A8A66F92E7B43CB867B074C6EF5098C, vertexId=vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA]
2016-05-18 00:58:12,047 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091820_0001_1][Event:VERTEX_FINISHED]: vertexName=D91C4B2E0DA44E8691A9A49137BDF7BA, vertexId=vertex_1463533091820_0001_1_00, initRequestedTime=1463533091979, initedTime=1463533091993, startRequestedTime=1463533091981, startedTime=1463533091996, finishTime=1463533092047, timeTaken=51, status=FAILED, diagnostics=Task failed, taskId=task_1463533091820_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463533091820_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]]
Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
Vertex vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA] killed/failed due to:OWN_TASK_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=1463533092002, firstTasksToStart=[ task_1463533091820_0001_1_00_000000 ], lastTaskFinishTime=1463533092043, lastTasksToFinish=[ task_1463533091820_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-18 00:58:12,047 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-18 00:58:12,048 INFO container.AMContainerImpl (AMContainerImpl.java:transition(704)) - Container container_1463533091820_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
2016-05-18 00:58:12,048 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091820_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533091820_0001_00_000001, stoppedTime=1463533092048, exitStatus=1
2016-05-18 00:58:12,048 INFO node.AMNodeImpl (AMNodeImpl.java:transition(298)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463533091820_0001_1_00_000000_0 failed: true container: container_1463533091820_0001_00_000001 numFailedTAs: 0
2016-05-18 00:58:12,048 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-18 00:58:12,048 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=VERTEX_FAILURE
2016-05-18 00:58:12,048 INFO impl.DAGImpl (DAGImpl.java:finishWithTerminationCause(1350)) - DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
2016-05-18 00:58:12,049 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533091820_0001_1, queueSize=0
2016-05-18 00:58:12,050 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533091820_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533091820_0001_1, startTime=1463533091978, finishTime=1463533092049, timeTaken=71, status=FAILED, diagnostics=Vertex failed, vertexName=D91C4B2E0DA44E8691A9A49137BDF7BA, vertexId=vertex_1463533091820_0001_1_00, diagnostics=[Task failed, taskId=task_1463533091820_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463533091820_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA] killed/failed due to:OWN_TASK_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0, counters=Counters: 4, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, AM_CPU_MILLISECONDS=180, AM_GC_TIME_MILLIS=0
2016-05-18 00:58:12,050 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3082)) - Ignoring multiple aborts for vertex: vertex_1463533091820_0001_1_00 [D91C4B2E0DA44E8691A9A49137BDF7BA]
2016-05-18 00:58:12,050 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533091820_0001_1 finished with state: FAILED
2016-05-18 00:58:12,050 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533091820_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:12 Completed Dag: dag_1463533091820_0001_1
2016-05-18 00:58:12,051 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533091820_0001_1, dagState=FAILED
2016-05-18 00:58:12,051 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533091820_0001_1_post
2016-05-18 00:58:12,051 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:12,051 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:12,051 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:12,051 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[FCA9FC9C950A4A5A95B9F58419CF7005/CE3EB7B5890742929ECDCB02297FFD3F] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL, with id=dag_1463533091820_0001_1
2016-05-18 00:58:12,051 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[FCA9FC9C950A4A5A95B9F58419CF7005/CE3EB7B5890742929ECDCB02297FFD3F] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL, with id=dag_1463533091820_0001_1
2016-05-18 00:58:12,063 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...eld'_RESULTS.0_VALUES_ALL, applicationId=application_1463533091820_0001
2016-05-18 00:58:12,065 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:12,065 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:12,066 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:12,066 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:12,066 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:12,066 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:12,066 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:12,066 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:12,067 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:12,067 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:12,067 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:12,067 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:12,067 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:12,069 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:12,073 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FCA9FC9C950A4A5A95B9F58419CF7005/CE3EB7B5890742929ECDCB02297FFD3F/.tez/application_1463533091820_0001
2016-05-18 00:58:12,074 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-18 00:58:12,074 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-18 00:58:12,075 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-18 00:58:12,075 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_ALL/_temporary
2016-05-18 00:58:12,075 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.275
2016-05-18 00:58:12,075 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_VALUES_ALL
2016-05-18 00:58:12,099 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:12,099 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:12,135 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-18 00:58:12,140 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-18 00:58:12,141 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:12,141 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:12,141 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:12,156 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:12,157 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_-1
2016-05-18 00:58:12,167 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:12,167 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:12,167 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_-1"]
2016-05-18 00:58:12,167 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:12,167 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:12,167 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:12,169 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...'field'_RESULTS.0_ARGS_-1
2016-05-18 00:58:12,169 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:12,182 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:12,186 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:12,186 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:12,187 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:12,195 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/A8EEC30400CC45B3A0589D674BBEECEC/B500EF798FEC4CCBB10859B4E6EAF6E2/.tez/application_1463533092186_0001 doesn't exist and is created
2016-05-18 00:58:12,209 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:12,209 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:12,210 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/A8EEC30400CC45B3A0589D674BBEECEC/B500EF798FEC4CCBB10859B4E6EAF6E2/.tez/application_1463533092186_0001_wd
2016-05-18 00:58:12,220 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533092186_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:12,220 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:12,221 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:12,222 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:12,223 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:12,223 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:12,223 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:12,224 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:12,226 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:12,226 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:12,226 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:12,226 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:12,227 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:12,227 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:12,227 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:12,227 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:12,228 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:12,228 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:12,229 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:12,229 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:12,231 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:12,232 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:12,232 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:12,232 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/A8EEC30400CC45B3A0589D674BBEECEC/B500EF798FEC4CCBB10859B4E6EAF6E2/.tez/application_1463533092186_0001_wd/localmode-log-dir
2016-05-18 00:58:12,232 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/A8EEC30400CC45B3A0589D674BBEECEC/B500EF798FEC4CCBB10859B4E6EAF6E2/.tez/application_1463533092186_0001_wd/localmode-log-dir/history.txt.appattempt_1463533092186_0001_000000, maxErrors=10
2016-05-18 00:58:12,233 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/A8EEC30400CC45B3A0589D674BBEECEC/B500EF798FEC4CCBB10859B4E6EAF6E2/.tez/application_1463533092186_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:12,233 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533092186_0001_000000, appSubmitTime=1463533092220, launchTime=1463533092220
2016-05-18 00:58:12,235 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:12,238 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:52653
2016-05-18 00:58:12,239 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533092186
2016-05-18 00:58:12,241 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533092186_0001_000000, startTime=1463533092241
2016-05-18 00:58:12,241 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:12,310 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:12,310 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:12,310 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...'field'_RESULTS.0_ARGS_-1, applicationId=application_1463533092186_0001, dagName=[A8EEC30400CC45B3A0589D674BBEECEC/B500EF798FEC4CCBB10859B4E6EAF6E2] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1
2016-05-18 00:58:12,325 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [A8EEC30400CC45B3A0589D674BBEECEC/B500EF798FEC4CCBB10859B4E6EAF6E2] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1
2016-05-18 00:58:12,325 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:12,326 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533092186_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/A8EEC30400CC45B3A0589D674BBEECEC/B500EF798FEC4CCBB10859B4E6EAF6E2/.tez/application_1463533092186_0001_wd/localmode-log-dir/dag_1463533092186_0001_1.dot
2016-05-18 00:58:12,327 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/A8EEC30400CC45B3A0589D674BBEECEC/B500EF798FEC4CCBB10859B4E6EAF6E2/.tez/application_1463533092186_0001_wd/localmode-log-dir/dag_1463533092186_0001_1-tez-dag.pb.txt
2016-05-18 00:58:12,330 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533092186_0001_1
2016-05-18 00:58:12,330 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:12,331 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [A8EEC30400CC45B3A0589D674BBEECEC/B500EF798FEC4CCBB10859B4E6EAF6E2] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1
2016-05-18 00:58:12 Running Dag: dag_1463533092186_0001_1
2016-05-18 00:58:12,339 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092186_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533092186_0001_1, submitTime=1463533092325
2016-05-18 00:58:12,340 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/A8EEC30400CC45B3A0589D674BBEECEC/B500EF798FEC4CCBB10859B4E6EAF6E2/.tez/application_1463533092186_0001/tez-conf.pb" } size: 36913 timestamp: 1463533092000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:12,340 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB]
2016-05-18 00:58:12,341 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB]
2016-05-18 00:58:12,341 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:12,341 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:12,342 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092186_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533092186_0001_1, initTime=1463533092341
2016-05-18 00:58:12,342 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533092186_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:12,342 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:12,342 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092186_0001_1][Event:DAG_STARTED]: dagID=dag_1463533092186_0001_1, startTime=1463533092342
2016-05-18 00:58:12,342 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533092186_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:12,343 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: ECBBE23838AB4F39A2DFA6ECEAE0CBDB : {77F7182E4AAC4991A6B9A76A9C1ABA74={InputName=77F7182E4AAC4991A6B9A76A9C1ABA74}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:12,343 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 77F7182E4AAC4991A6B9A76A9C1ABA74, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:12,343 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB]
2016-05-18 00:58:12,343 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB]
2016-05-18 00:58:12,343 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB]
2016-05-18 00:58:12,343 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB]
2016-05-18 00:58:12,344 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...'field'_RESULTS.0_ARGS_-1, applicationId=application_1463533092186_0001, dagName=[A8EEC30400CC45B3A0589D674BBEECEC/B500EF798FEC4CCBB10859B4E6EAF6E2] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1
2016-05-18 00:58:12,344 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:12,345 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 77F7182E4AAC4991A6B9A76A9C1ABA74 on vertex vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB]
2016-05-18 00:58:12,347 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 77F7182E4AAC4991A6B9A76A9C1ABA74 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:12,348 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:12,349 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:12,349 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:12,349 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:12,350 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 77F7182E4AAC4991A6B9A76A9C1ABA74 on vertex vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB]
2016-05-18 00:58:12,353 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:12,353 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {77F7182E4AAC4991A6B9A76A9C1ABA74=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:12,354 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB] parallelism set to 1
2016-05-18 00:58:12,354 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:12,355 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB], numAdditionalOutputs=1
2016-05-18 00:58:12,355 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=41EEF1D591FD46C1897C6FFE78CCE10E, vertex=vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:12,368 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for ECBBE23838AB4F39A2DFA6ECEAE0CBDB:41EEF1D591FD46C1897C6FFE78CCE10E using oldmapred API
2016-05-18 00:58:12,368 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=41EEF1D591FD46C1897C6FFE78CCE10E, vertexName=ECBBE23838AB4F39A2DFA6ECEAE0CBDB, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:12,369 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092186_0001_1][Event:VERTEX_INITIALIZED]: vertexName=ECBBE23838AB4F39A2DFA6ECEAE0CBDB, vertexId=vertex_1463533092186_0001_1_00, initRequestedTime=1463533092342, initedTime=1463533092355, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:12,369 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB] with distanceFromRoot: 0
2016-05-18 00:58:12,370 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:12,370 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092186_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533092186_0001_1_00, startRequestedTime=1463533092344, startedTime=1463533092370
2016-05-18 00:58:12,370 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092186_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533092186_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:12,371 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:12,371 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in ECBBE23838AB4F39A2DFA6ECEAE0CBDB
2016-05-18 00:58:12,371 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB]
2016-05-18 00:58:12,371 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092186_0001_1][Event:TASK_STARTED]: vertexName=ECBBE23838AB4F39A2DFA6ECEAE0CBDB, taskId=task_1463533092186_0001_1_00_000000, scheduledTime=1463533092371, launchTime=1463533092371
2016-05-18 00:58:12,372 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB]
2016-05-18 00:58:12,373 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:12,375 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533092186_0001_00_000001, launchTime=1463533092374
2016-05-18 00:58:12,375 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533092186_0001_00_000001
2016-05-18 00:58:12,376 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533092186_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533092186_0001_1_00_000000_0
2016-05-18 00:58:12,376 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533092186_0001_1_00_000000_0] started. Is using containerId: [container_1463533092186_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:12,376 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092186_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=ECBBE23838AB4F39A2DFA6ECEAE0CBDB, taskAttemptId=attempt_1463533092186_0001_1_00_000000_0, startTime=1463533092376, containerId=container_1463533092186_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:12 Starting to run new task attempt: attempt_1463533092186_0001_1_00_000000_0
2016-05-18 00:58:12,377 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533092186_0001_1_00_000000_0
2016-05-18 00:58:12,377 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:12,377 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:12,377 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A8EEC30400CC45B3A0589D674BBEECEC/B500EF798FEC4CCBB10859B4E6EAF6E2] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1, VertexName: ECBBE23838AB4F39A2DFA6ECEAE0CBDB, VertexParallelism: 1, TaskAttemptID:attempt_1463533092186_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=77F7182E4AAC4991A6B9A76A9C1ABA74, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=41EEF1D591FD46C1897C6FFE78CCE10E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:12,377 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:12,378 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533092186_0001_1_00_000000_0
2016-05-18 00:58:12,378 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533092186_0001, with dag id: dag_1463533092186_0001_1
2016-05-18 00:58:12,378 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533092186_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:12,381 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533092186_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:12,385 INFO input.MRInput (MRInput.java:initialize(440)) - 77F7182E4AAC4991A6B9A76A9C1ABA74 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:12,386 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 77F7182E4AAC4991A6B9A76A9C1ABA74
2016-05-18 00:58:12,386 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:12,392 INFO output.MROutput (MROutput.java:initialize(421)) - 41EEF1D591FD46C1897C6FFE78CCE10E: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:12,393 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: ECBBE23838AB4F39A2DFA6ECEAE0CBDB, ordinal: 0
2016-05-18 00:58:12,393 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ECBBE23838AB4F39A2DFA6ECEAE0CBDB, mem on start (mb), free: 299, total: 383, max: 672
2016-05-18 00:58:12,393 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:12,393 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:12,394 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:12,394 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:12,394 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[77F7182E4AAC4991A6B9A76A9C1ABA74:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [41EEF1D591FD46C1897C6FFE78CCE10E:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:12,394 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:12,395 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:12,395 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:12,395 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[77F7182E4AAC4991A6B9A76A9C1ABA74:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [41EEF1D591FD46C1897C6FFE78CCE10E:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:12,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:12,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 77F7182E4AAC4991A6B9A76A9C1ABA74 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:12,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 77F7182E4AAC4991A6B9A76A9C1ABA74
2016-05-18 00:58:12,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:12,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:12,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:12,397 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533092186_0001_1_00_000000_0
2016-05-18 00:58:12,399 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:12,400 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:12,400 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 77F7182E4AAC4991A6B9A76A9C1ABA74 initialized RecordReader from event
2016-05-18 00:58:12,402 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 77F7182E4AAC4991A6B9A76A9C1ABA74
2016-05-18 00:58:12,402 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_-1"], id: 41EEF1D591FD46C1897C6FFE78CCE10E
2016-05-18 00:58:12,402 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_-1"]
2016-05-18 00:58:12,402 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:12,403 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: ECBBE23838AB4F39A2DFA6ECEAE0CBDB, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:12,406 INFO output.MROutput (MROutput.java:close(526)) - 41EEF1D591FD46C1897C6FFE78CCE10E closed
2016-05-18 00:58:12,407 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533092186_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:12,409 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533092186_0001_1_00_000000_0
2016-05-18 00:58:12,410 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ECBBE23838AB4F39A2DFA6ECEAE0CBDB, mem on close (mb), free: 297, total: 383, max: 672
2016-05-18 00:58:12,410 INFO output.MROutput (MROutput.java:close(526)) - 41EEF1D591FD46C1897C6FFE78CCE10E closed
2016-05-18 00:58:12,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:12 Completed running task attempt: attempt_1463533092186_0001_1_00_000000_0
2016-05-18 00:58:12,410 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533092186_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:12,411 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533092186_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:12,411 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533092186_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533092403, Process_Duration=6, Process_End_Time=1463533092409, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:12,413 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533092186_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:12,413 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092186_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=ECBBE23838AB4F39A2DFA6ECEAE0CBDB, taskAttemptId=attempt_1463533092186_0001_1_00_000000_0, creationTime=1463533092372, allocationTime=1463533092374, startTime=1463533092376, finishTime=1463533092413, timeTaken=37, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:12,413 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:12,413 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:12,413 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533092186_0001_1_00_000000_0
2016-05-18 00:58:12,414 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092186_0001_1][Event:TASK_FINISHED]: vertexName=ECBBE23838AB4F39A2DFA6ECEAE0CBDB, taskId=task_1463533092186_0001_1_00_000000, startTime=1463533092376, finishTime=1463533092413, timeTaken=37, status=SUCCEEDED, successfulAttemptID=attempt_1463533092186_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533092403, Process_Duration=6, Process_End_Time=1463533092409, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:12,414 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:12,415 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:12,415 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB]
2016-05-18 00:58:12,416 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092186_0001_1][Event:VERTEX_FINISHED]: vertexName=ECBBE23838AB4F39A2DFA6ECEAE0CBDB, vertexId=vertex_1463533092186_0001_1_00, initRequestedTime=1463533092342, initedTime=1463533092355, startRequestedTime=1463533092344, startedTime=1463533092370, finishTime=1463533092416, timeTaken=46, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533092403, Process_Duration=6, Process_End_Time=1463533092409, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533092376, firstTasksToStart=[ task_1463533092186_0001_1_00_000000 ], lastTaskFinishTime=1463533092413, lastTasksToFinish=[ task_1463533092186_0001_1_00_000000 ], minTaskDuration=37, maxTaskDuration=37, avgTaskDuration=37.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533092186_0001_1_00_000000 ], longestDurationTasks=[ task_1463533092186_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:12,417 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:12,417 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:12,417 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:12,417 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533092186_0001_1
2016-05-18 00:58:12,417 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092186_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533092186_0001_1
2016-05-18 00:58:12,417 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 41EEF1D591FD46C1897C6FFE78CCE10E for vertex: vertex_1463533092186_0001_1_00 [ECBBE23838AB4F39A2DFA6ECEAE0CBDB], outputName: 41EEF1D591FD46C1897C6FFE78CCE10E
2016-05-18 00:58:12,417 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533092186_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:12,418 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533092186_0001_00_000001
2016-05-18 00:58:12,419 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092186_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533092186_0001_00_000001, stoppedTime=1463533092419, exitStatus=0
2016-05-18 00:58:12,420 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533092186_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:12,425 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:41EEF1D591FD46C1897C6FFE78CCE10E of vertex/vertexGroup:ECBBE23838AB4F39A2DFA6ECEAE0CBDB isVertexGroupOutput:false
2016-05-18 00:58:12,425 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:12,427 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533092186_0001_1, queueSize=0
2016-05-18 00:58:12,427 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092186_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533092186_0001_1, startTime=1463533092342, finishTime=1463533092425, timeTaken=83, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=190, AM_GC_TIME_MILLIS=10, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533092403, Process_Duration=6, Process_End_Time=1463533092409, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:12,427 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533092186_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:12,428 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533092186_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:12 Completed Dag: dag_1463533092186_0001_1
2016-05-18 00:58:12,428 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533092186_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:12,428 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533092186_0001_1_post
2016-05-18 00:58:12,428 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:12,428 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533092186_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:12,428 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:12,428 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:12,428 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533092186_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:12,429 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533092186_0001_00_000001, Exiting
2016-05-18 00:58:12,429 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533092186_0001_00_000001
2016-05-18 00:58:12,429 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[A8EEC30400CC45B3A0589D674BBEECEC/B500EF798FEC4CCBB10859B4E6EAF6E2] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1, with id=dag_1463533092186_0001_1
2016-05-18 00:58:12,429 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533092186_0001_00_000001
2016-05-18 00:58:12,429 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[A8EEC30400CC45B3A0589D674BBEECEC/B500EF798FEC4CCBB10859B4E6EAF6E2] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1, with id=dag_1463533092186_0001_1
2016-05-18 00:58:12,433 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533092186_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:12,433 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533092186_0001_00_000001 completed successfully
2016-05-18 00:58:12,434 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:58:12,434 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:58:12,438 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...'field'_RESULTS.0_ARGS_-1, applicationId=application_1463533092186_0001
2016-05-18 00:58:12,441 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:12,441 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:12,441 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:12,441 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:12,441 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:12,442 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:12,445 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:12,445 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:12,445 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:12,446 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:12,446 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:12,447 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:12,447 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:12,448 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:12,452 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/A8EEC30400CC45B3A0589D674BBEECEC/B500EF798FEC4CCBB10859B4E6EAF6E2/.tez/application_1463533092186_0001
2016-05-18 00:58:12,453 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_-1/_temporary
2016-05-18 00:58:12,454 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.286
2016-05-18 00:58:12,533 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_ARGS_-1
2016-05-18 00:58:12,555 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:12,556 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:12,584 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-18 00:58:12,589 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.008
2016-05-18 00:58:12,590 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 00:58:12,590 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 00:58:12,591 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_ARGS_'field2'
2016-05-18 00:58:12,618 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:12,618 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:12,653 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-18 00:58:12,659 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-18 00:58:12,659 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:12,659 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:12,660 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:12,676 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:12,676 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_RESULTS
2016-05-18 00:58:12,686 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:12,686 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:12,686 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_RESULTS"]
2016-05-18 00:58:12,686 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:12,687 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:12,687 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:12,687 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...d'_RESULTS.0_ARGS_RESULTS
2016-05-18 00:58:12,687 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:12,700 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:12,704 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:12,705 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:12,705 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:12,714 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F60D02CF42454C67809FB20BC4FFE2D6/BD2B5691B17D4B12BD8465C203DDA342/.tez/application_1463533092704_0001 doesn't exist and is created
2016-05-18 00:58:12,728 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:12,728 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:12,729 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F60D02CF42454C67809FB20BC4FFE2D6/BD2B5691B17D4B12BD8465C203DDA342/.tez/application_1463533092704_0001_wd
2016-05-18 00:58:12,738 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533092704_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:12,738 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:12,738 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:12,740 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:12,740 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:12,740 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:12,740 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:12,741 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:12,742 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:12,742 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:12,743 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:12,743 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:12,743 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:12,743 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:12,743 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:12,743 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:12,744 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:12,744 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:12,744 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:12,744 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:12,746 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:12,747 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:12,747 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:12,748 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F60D02CF42454C67809FB20BC4FFE2D6/BD2B5691B17D4B12BD8465C203DDA342/.tez/application_1463533092704_0001_wd/localmode-log-dir
2016-05-18 00:58:12,748 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F60D02CF42454C67809FB20BC4FFE2D6/BD2B5691B17D4B12BD8465C203DDA342/.tez/application_1463533092704_0001_wd/localmode-log-dir/history.txt.appattempt_1463533092704_0001_000000, maxErrors=10
2016-05-18 00:58:12,748 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F60D02CF42454C67809FB20BC4FFE2D6/BD2B5691B17D4B12BD8465C203DDA342/.tez/application_1463533092704_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:12,749 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533092704_0001_000000, appSubmitTime=1463533092737, launchTime=1463533092737
2016-05-18 00:58:12,751 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:12,754 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:34410
2016-05-18 00:58:12,755 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533092704
2016-05-18 00:58:12,756 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533092704_0001_000000, startTime=1463533092756
2016-05-18 00:58:12,756 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:12,829 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:12,829 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:12,829 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...d'_RESULTS.0_ARGS_RESULTS, applicationId=application_1463533092704_0001, dagName=[F60D02CF42454C67809FB20BC4FFE2D6/BD2B5691B17D4B12BD8465C203DDA342] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS
2016-05-18 00:58:12,844 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [F60D02CF42454C67809FB20BC4FFE2D6/BD2B5691B17D4B12BD8465C203DDA342] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS
2016-05-18 00:58:12,845 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:12,845 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533092704_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F60D02CF42454C67809FB20BC4FFE2D6/BD2B5691B17D4B12BD8465C203DDA342/.tez/application_1463533092704_0001_wd/localmode-log-dir/dag_1463533092704_0001_1.dot
2016-05-18 00:58:12,846 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F60D02CF42454C67809FB20BC4FFE2D6/BD2B5691B17D4B12BD8465C203DDA342/.tez/application_1463533092704_0001_wd/localmode-log-dir/dag_1463533092704_0001_1-tez-dag.pb.txt
2016-05-18 00:58:12,849 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533092704_0001_1
2016-05-18 00:58:12,849 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:12,849 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [F60D02CF42454C67809FB20BC4FFE2D6/BD2B5691B17D4B12BD8465C203DDA342] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS
2016-05-18 00:58:12 Running Dag: dag_1463533092704_0001_1
2016-05-18 00:58:12,858 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092704_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533092704_0001_1, submitTime=1463533092845
2016-05-18 00:58:12,859 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F60D02CF42454C67809FB20BC4FFE2D6/BD2B5691B17D4B12BD8465C203DDA342/.tez/application_1463533092704_0001/tez-conf.pb" } size: 36913 timestamp: 1463533092000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:12,859 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F]
2016-05-18 00:58:12,859 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F]
2016-05-18 00:58:12,859 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:12,859 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:12,860 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092704_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533092704_0001_1, initTime=1463533092859
2016-05-18 00:58:12,860 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533092704_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:12,860 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:12,861 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...d'_RESULTS.0_ARGS_RESULTS, applicationId=application_1463533092704_0001, dagName=[F60D02CF42454C67809FB20BC4FFE2D6/BD2B5691B17D4B12BD8465C203DDA342] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS
2016-05-18 00:58:12,861 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092704_0001_1][Event:DAG_STARTED]: dagID=dag_1463533092704_0001_1, startTime=1463533092861
2016-05-18 00:58:12,861 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533092704_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:12,861 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: D8FD10616A6D46BBAA7CDF8FE20BCC7F : {AB1792BA2C574A2D853048242D9DB81E={InputName=AB1792BA2C574A2D853048242D9DB81E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:12,861 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: AB1792BA2C574A2D853048242D9DB81E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:12,861 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F]
2016-05-18 00:58:12,861 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F]
2016-05-18 00:58:12,861 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F]
2016-05-18 00:58:12,862 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F]
2016-05-18 00:58:12,862 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:12,864 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: AB1792BA2C574A2D853048242D9DB81E on vertex vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F]
2016-05-18 00:58:12,866 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input AB1792BA2C574A2D853048242D9DB81E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:12,867 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:12,867 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:12,868 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:12,868 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:12,869 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: AB1792BA2C574A2D853048242D9DB81E on vertex vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F]
2016-05-18 00:58:12,869 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:12,870 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {AB1792BA2C574A2D853048242D9DB81E=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:12,871 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F] parallelism set to 1
2016-05-18 00:58:12,872 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:12,872 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F], numAdditionalOutputs=1
2016-05-18 00:58:12,872 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=90E15BB6BFDC4F2BA4CE7DCDDD07BE27, vertex=vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:12,873 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D8FD10616A6D46BBAA7CDF8FE20BCC7F:90E15BB6BFDC4F2BA4CE7DCDDD07BE27 using oldmapred API
2016-05-18 00:58:12,873 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=90E15BB6BFDC4F2BA4CE7DCDDD07BE27, vertexName=D8FD10616A6D46BBAA7CDF8FE20BCC7F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:12,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092704_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D8FD10616A6D46BBAA7CDF8FE20BCC7F, vertexId=vertex_1463533092704_0001_1_00, initRequestedTime=1463533092861, initedTime=1463533092872, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:12,874 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F] with distanceFromRoot: 0
2016-05-18 00:58:12,874 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:12,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092704_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533092704_0001_1_00, startRequestedTime=1463533092862, startedTime=1463533092874
2016-05-18 00:58:12,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092704_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533092704_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:12,875 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:12,875 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in D8FD10616A6D46BBAA7CDF8FE20BCC7F
2016-05-18 00:58:12,875 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F]
2016-05-18 00:58:12,876 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092704_0001_1][Event:TASK_STARTED]: vertexName=D8FD10616A6D46BBAA7CDF8FE20BCC7F, taskId=task_1463533092704_0001_1_00_000000, scheduledTime=1463533092876, launchTime=1463533092876
2016-05-18 00:58:12,876 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F]
2016-05-18 00:58:12,877 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:12,878 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533092704_0001_00_000001, launchTime=1463533092878
2016-05-18 00:58:12,879 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533092704_0001_00_000001
2016-05-18 00:58:12,879 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533092704_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533092704_0001_1_00_000000_0
2016-05-18 00:58:12,880 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533092704_0001_1_00_000000_0] started. Is using containerId: [container_1463533092704_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:12,880 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092704_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D8FD10616A6D46BBAA7CDF8FE20BCC7F, taskAttemptId=attempt_1463533092704_0001_1_00_000000_0, startTime=1463533092879, containerId=container_1463533092704_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:12 Starting to run new task attempt: attempt_1463533092704_0001_1_00_000000_0
2016-05-18 00:58:12,881 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533092704_0001_1_00_000000_0
2016-05-18 00:58:12,881 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:12,881 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:12,882 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [F60D02CF42454C67809FB20BC4FFE2D6/BD2B5691B17D4B12BD8465C203DDA342] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS, VertexName: D8FD10616A6D46BBAA7CDF8FE20BCC7F, VertexParallelism: 1, TaskAttemptID:attempt_1463533092704_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AB1792BA2C574A2D853048242D9DB81E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=90E15BB6BFDC4F2BA4CE7DCDDD07BE27, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:12,882 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:12,882 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533092704_0001_1_00_000000_0
2016-05-18 00:58:12,883 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533092704_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:12,883 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533092704_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:12,885 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533092704_0001, with dag id: dag_1463533092704_0001_1
2016-05-18 00:58:12,886 INFO input.MRInput (MRInput.java:initialize(440)) - AB1792BA2C574A2D853048242D9DB81E using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:12,886 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: AB1792BA2C574A2D853048242D9DB81E
2016-05-18 00:58:12,887 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:12,895 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D8FD10616A6D46BBAA7CDF8FE20BCC7F, ordinal: 0
2016-05-18 00:58:12,896 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D8FD10616A6D46BBAA7CDF8FE20BCC7F, mem on start (mb), free: 192, total: 383, max: 672
2016-05-18 00:58:12,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:12,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:12,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:12,897 INFO output.MROutput (MROutput.java:initialize(421)) - 90E15BB6BFDC4F2BA4CE7DCDDD07BE27: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:12,898 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:12,898 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[AB1792BA2C574A2D853048242D9DB81E:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [90E15BB6BFDC4F2BA4CE7DCDDD07BE27:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:12,898 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:12,898 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:12,898 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:12,899 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[AB1792BA2C574A2D853048242D9DB81E:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [90E15BB6BFDC4F2BA4CE7DCDDD07BE27:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:12,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:12,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: AB1792BA2C574A2D853048242D9DB81E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:12,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: AB1792BA2C574A2D853048242D9DB81E
2016-05-18 00:58:12,900 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:12,900 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:12,900 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:12,901 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533092704_0001_1_00_000000_0
2016-05-18 00:58:12,904 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:12,904 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: AB1792BA2C574A2D853048242D9DB81E
2016-05-18 00:58:12,904 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_RESULTS"], id: 90E15BB6BFDC4F2BA4CE7DCDDD07BE27
2016-05-18 00:58:12,905 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_RESULTS"]
2016-05-18 00:58:12,905 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:12,906 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:12,906 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - AB1792BA2C574A2D853048242D9DB81E initialized RecordReader from event
2016-05-18 00:58:12,906 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D8FD10616A6D46BBAA7CDF8FE20BCC7F, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:12,909 INFO output.MROutput (MROutput.java:close(526)) - 90E15BB6BFDC4F2BA4CE7DCDDD07BE27 closed
2016-05-18 00:58:12,909 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533092704_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:12,910 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533092704_0001_1_00_000000_0
2016-05-18 00:58:12,910 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D8FD10616A6D46BBAA7CDF8FE20BCC7F, mem on close (mb), free: 190, total: 383, max: 672
2016-05-18 00:58:12,910 INFO output.MROutput (MROutput.java:close(526)) - 90E15BB6BFDC4F2BA4CE7DCDDD07BE27 closed
2016-05-18 00:58:12,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:12 Completed running task attempt: attempt_1463533092704_0001_1_00_000000_0
2016-05-18 00:58:12,911 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533092704_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:12,911 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533092704_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:12,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533092704_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533092907, Process_Duration=3, Process_End_Time=1463533092910, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:12,913 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533092704_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:12,913 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:12,913 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:12,913 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:12,913 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533092704_0001_00_000001
2016-05-18 00:58:12,913 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092704_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D8FD10616A6D46BBAA7CDF8FE20BCC7F, taskAttemptId=attempt_1463533092704_0001_1_00_000000_0, creationTime=1463533092876, allocationTime=1463533092878, startTime=1463533092879, finishTime=1463533092913, timeTaken=34, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:12,914 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533092704_0001_1_00_000000_0
2016-05-18 00:58:12,915 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092704_0001_1][Event:TASK_FINISHED]: vertexName=D8FD10616A6D46BBAA7CDF8FE20BCC7F, taskId=task_1463533092704_0001_1_00_000000, startTime=1463533092879, finishTime=1463533092914, timeTaken=35, status=SUCCEEDED, successfulAttemptID=attempt_1463533092704_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533092907, Process_Duration=3, Process_End_Time=1463533092910, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:12,915 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:12,915 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F]
2016-05-18 00:58:12,916 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092704_0001_1][Event:VERTEX_FINISHED]: vertexName=D8FD10616A6D46BBAA7CDF8FE20BCC7F, vertexId=vertex_1463533092704_0001_1_00, initRequestedTime=1463533092861, initedTime=1463533092872, startRequestedTime=1463533092862, startedTime=1463533092874, finishTime=1463533092915, timeTaken=41, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533092907, Process_Duration=3, Process_End_Time=1463533092910, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533092879, firstTasksToStart=[ task_1463533092704_0001_1_00_000000 ], lastTaskFinishTime=1463533092913, lastTasksToFinish=[ task_1463533092704_0001_1_00_000000 ], minTaskDuration=34, maxTaskDuration=34, avgTaskDuration=34.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533092704_0001_1_00_000000 ], longestDurationTasks=[ task_1463533092704_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:12,916 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:12,916 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092704_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533092704_0001_00_000001, stoppedTime=1463533092916, exitStatus=0
2016-05-18 00:58:12,916 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:12,916 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:12,917 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533092704_0001_1
2016-05-18 00:58:12,917 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092704_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533092704_0001_1
2016-05-18 00:58:12,917 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 90E15BB6BFDC4F2BA4CE7DCDDD07BE27 for vertex: vertex_1463533092704_0001_1_00 [D8FD10616A6D46BBAA7CDF8FE20BCC7F], outputName: 90E15BB6BFDC4F2BA4CE7DCDDD07BE27
2016-05-18 00:58:12,917 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533092704_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:12,918 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533092704_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:12,923 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:90E15BB6BFDC4F2BA4CE7DCDDD07BE27 of vertex/vertexGroup:D8FD10616A6D46BBAA7CDF8FE20BCC7F isVertexGroupOutput:false
2016-05-18 00:58:12,923 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:12,924 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533092704_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:12,924 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533092704_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:12,925 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533092704_0001_00_000001, Exiting
2016-05-18 00:58:12,925 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533092704_0001_00_000001
2016-05-18 00:58:12,925 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533092704_0001_00_000001
2016-05-18 00:58:12,925 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533092704_0001_1, queueSize=0
2016-05-18 00:58:12,925 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533092704_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:12,926 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533092704_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533092704_0001_1, startTime=1463533092860, finishTime=1463533092923, timeTaken=63, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=150, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533092907, Process_Duration=3, Process_End_Time=1463533092910, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:12,926 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533092704_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:12,926 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533092704_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:12 Completed Dag: dag_1463533092704_0001_1
2016-05-18 00:58:12,926 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533092704_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:12,926 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533092704_0001_1_post
2016-05-18 00:58:12,926 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:12,926 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:12,927 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:12,927 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[F60D02CF42454C67809FB20BC4FFE2D6/BD2B5691B17D4B12BD8465C203DDA342] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS, with id=dag_1463533092704_0001_1
2016-05-18 00:58:12,926 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533092704_0001_00_000001 completed successfully
2016-05-18 00:58:12,927 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[F60D02CF42454C67809FB20BC4FFE2D6/BD2B5691B17D4B12BD8465C203DDA342] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS, with id=dag_1463533092704_0001_1
2016-05-18 00:58:12,935 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...d'_RESULTS.0_ARGS_RESULTS, applicationId=application_1463533092704_0001
2016-05-18 00:58:12,937 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:12,937 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:12,937 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:12,937 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:12,937 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:12,937 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:12,938 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:12,938 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:12,938 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:12,938 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:12,938 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:12,939 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:12,939 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:12,940 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:12,942 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F60D02CF42454C67809FB20BC4FFE2D6/BD2B5691B17D4B12BD8465C203DDA342/.tez/application_1463533092704_0001
2016-05-18 00:58:12,943 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_RESULTS/_temporary
2016-05-18 00:58:12,943 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.256
2016-05-18 00:58:12,972 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:58:12,972 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:58:13,012 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_ARGS_RESULTS
2016-05-18 00:58:13,036 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:13,036 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:13,069 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-05-18 00:58:13,076 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.013
2016-05-18 00:58:13,077 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 00:58:13,077 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 00:58:13,077 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_ARGS_ALL
2016-05-18 00:58:13,111 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:13,111 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:13,165 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-18 00:58:13,167 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-18 00:58:13,168 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:13,168 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:13,169 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:13,190 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:13,190 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_-1
2016-05-18 00:58:13,203 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:13,203 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:13,204 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_-1"]
2016-05-18 00:58:13,204 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:13,204 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:13,204 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:13,206 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...'_RESULTS.ALL_'field2'_-1
2016-05-18 00:58:13,206 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:13,220 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:13,224 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:13,225 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:13,225 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:13,234 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/03E175CC08FA43E48FD93993011AC086/ED07B0DBD2BD4CCB89F709B65F7D4EEB/.tez/application_1463533093224_0001 doesn't exist and is created
2016-05-18 00:58:13,248 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:13,248 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:13,249 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/03E175CC08FA43E48FD93993011AC086/ED07B0DBD2BD4CCB89F709B65F7D4EEB/.tez/application_1463533093224_0001_wd
2016-05-18 00:58:13,259 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533093224_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:13,259 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:13,259 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:13,262 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:13,262 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:13,262 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:13,262 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:13,264 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:13,265 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:13,265 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:13,266 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:13,266 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:13,267 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:13,267 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:13,267 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:13,267 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:13,268 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:13,268 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:13,268 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:13,268 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:13,269 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:13,270 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:13,270 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:13,270 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/03E175CC08FA43E48FD93993011AC086/ED07B0DBD2BD4CCB89F709B65F7D4EEB/.tez/application_1463533093224_0001_wd/localmode-log-dir
2016-05-18 00:58:13,271 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/03E175CC08FA43E48FD93993011AC086/ED07B0DBD2BD4CCB89F709B65F7D4EEB/.tez/application_1463533093224_0001_wd/localmode-log-dir/history.txt.appattempt_1463533093224_0001_000000, maxErrors=10
2016-05-18 00:58:13,271 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/03E175CC08FA43E48FD93993011AC086/ED07B0DBD2BD4CCB89F709B65F7D4EEB/.tez/application_1463533093224_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:13,271 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533093224_0001_000000, appSubmitTime=1463533093258, launchTime=1463533093259
2016-05-18 00:58:13,280 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:13,283 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:39329
2016-05-18 00:58:13,283 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533093224
2016-05-18 00:58:13,287 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533093224_0001_000000, startTime=1463533093287
2016-05-18 00:58:13,287 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:13,348 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:13,348 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:13,349 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...'_RESULTS.ALL_'field2'_-1, applicationId=application_1463533093224_0001, dagName=[03E175CC08FA43E48FD93993011AC086/ED07B0DBD2BD4CCB89F709B65F7D4EEB] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1
2016-05-18 00:58:13,376 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [03E175CC08FA43E48FD93993011AC086/ED07B0DBD2BD4CCB89F709B65F7D4EEB] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1
2016-05-18 00:58:13,376 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:13,377 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533093224_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/03E175CC08FA43E48FD93993011AC086/ED07B0DBD2BD4CCB89F709B65F7D4EEB/.tez/application_1463533093224_0001_wd/localmode-log-dir/dag_1463533093224_0001_1.dot
2016-05-18 00:58:13,377 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/03E175CC08FA43E48FD93993011AC086/ED07B0DBD2BD4CCB89F709B65F7D4EEB/.tez/application_1463533093224_0001_wd/localmode-log-dir/dag_1463533093224_0001_1-tez-dag.pb.txt
2016-05-18 00:58:13,381 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533093224_0001_1
2016-05-18 00:58:13,381 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:13,381 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [03E175CC08FA43E48FD93993011AC086/ED07B0DBD2BD4CCB89F709B65F7D4EEB] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1
2016-05-18 00:58:13 Running Dag: dag_1463533093224_0001_1
2016-05-18 00:58:13,390 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093224_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533093224_0001_1, submitTime=1463533093376
2016-05-18 00:58:13,391 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/03E175CC08FA43E48FD93993011AC086/ED07B0DBD2BD4CCB89F709B65F7D4EEB/.tez/application_1463533093224_0001/tez-conf.pb" } size: 36913 timestamp: 1463533093000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:13,392 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0]
2016-05-18 00:58:13,392 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0]
2016-05-18 00:58:13,392 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:13,392 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:13,393 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093224_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533093224_0001_1, initTime=1463533093392
2016-05-18 00:58:13,393 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533093224_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:13,393 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:13,393 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093224_0001_1][Event:DAG_STARTED]: dagID=dag_1463533093224_0001_1, startTime=1463533093393
2016-05-18 00:58:13,393 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533093224_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:13,393 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: DAA31E5D794B4CC886049CBE7575C4F0 : {5F19CE8E2297420C955E344E29BD22B4={InputName=5F19CE8E2297420C955E344E29BD22B4}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:13,393 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 5F19CE8E2297420C955E344E29BD22B4, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:13,393 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0]
2016-05-18 00:58:13,394 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0]
2016-05-18 00:58:13,394 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0]
2016-05-18 00:58:13,394 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0]
2016-05-18 00:58:13,395 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:13,395 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...'_RESULTS.ALL_'field2'_-1, applicationId=application_1463533093224_0001, dagName=[03E175CC08FA43E48FD93993011AC086/ED07B0DBD2BD4CCB89F709B65F7D4EEB] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1
2016-05-18 00:58:13,396 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 5F19CE8E2297420C955E344E29BD22B4 on vertex vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0]
2016-05-18 00:58:13,398 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 5F19CE8E2297420C955E344E29BD22B4 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:13,400 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:13,400 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:13,401 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:13,401 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:13,401 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 5F19CE8E2297420C955E344E29BD22B4 on vertex vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0]
2016-05-18 00:58:13,402 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:13,403 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {5F19CE8E2297420C955E344E29BD22B4=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:13,403 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0] parallelism set to 1
2016-05-18 00:58:13,403 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:13,403 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0], numAdditionalOutputs=1
2016-05-18 00:58:13,404 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=42E95F5B438D456A81A3351F8C1E7B75, vertex=vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:13,406 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for DAA31E5D794B4CC886049CBE7575C4F0:42E95F5B438D456A81A3351F8C1E7B75 using oldmapred API
2016-05-18 00:58:13,406 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=42E95F5B438D456A81A3351F8C1E7B75, vertexName=DAA31E5D794B4CC886049CBE7575C4F0, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:13,407 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093224_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DAA31E5D794B4CC886049CBE7575C4F0, vertexId=vertex_1463533093224_0001_1_00, initRequestedTime=1463533093393, initedTime=1463533093403, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:13,407 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0] with distanceFromRoot: 0
2016-05-18 00:58:13,407 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:13,407 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093224_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533093224_0001_1_00, startRequestedTime=1463533093395, startedTime=1463533093407
2016-05-18 00:58:13,407 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093224_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533093224_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:13,407 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:13,408 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in DAA31E5D794B4CC886049CBE7575C4F0
2016-05-18 00:58:13,408 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0]
2016-05-18 00:58:13,408 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093224_0001_1][Event:TASK_STARTED]: vertexName=DAA31E5D794B4CC886049CBE7575C4F0, taskId=task_1463533093224_0001_1_00_000000, scheduledTime=1463533093408, launchTime=1463533093408
2016-05-18 00:58:13,408 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0]
2016-05-18 00:58:13,409 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:13,411 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533093224_0001, with dag id: dag_1463533093224_0001_1
2016-05-18 00:58:13,413 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533093224_0001_00_000001, launchTime=1463533093411
2016-05-18 00:58:13,414 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533093224_0001_00_000001
2016-05-18 00:58:13,415 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533093224_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533093224_0001_1_00_000000_0
2016-05-18 00:58:13,415 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533093224_0001_1_00_000000_0] started. Is using containerId: [container_1463533093224_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:13,415 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093224_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DAA31E5D794B4CC886049CBE7575C4F0, taskAttemptId=attempt_1463533093224_0001_1_00_000000_0, startTime=1463533093415, containerId=container_1463533093224_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:13 Starting to run new task attempt: attempt_1463533093224_0001_1_00_000000_0
2016-05-18 00:58:13,416 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533093224_0001_1_00_000000_0
2016-05-18 00:58:13,417 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:13,417 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:13,417 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [03E175CC08FA43E48FD93993011AC086/ED07B0DBD2BD4CCB89F709B65F7D4EEB] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1, VertexName: DAA31E5D794B4CC886049CBE7575C4F0, VertexParallelism: 1, TaskAttemptID:attempt_1463533093224_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5F19CE8E2297420C955E344E29BD22B4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=42E95F5B438D456A81A3351F8C1E7B75, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:13,417 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:13,418 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533093224_0001_1_00_000000_0
2016-05-18 00:58:13,418 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533093224_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:13,419 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533093224_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:13,420 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:13,421 INFO input.MRInput (MRInput.java:initialize(440)) - 5F19CE8E2297420C955E344E29BD22B4 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:13,421 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 5F19CE8E2297420C955E344E29BD22B4
2016-05-18 00:58:13,427 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DAA31E5D794B4CC886049CBE7575C4F0, ordinal: 0
2016-05-18 00:58:13,427 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DAA31E5D794B4CC886049CBE7575C4F0, mem on start (mb), free: 286, total: 383, max: 672
2016-05-18 00:58:13,427 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:13,427 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:13,427 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:13,432 INFO output.MROutput (MROutput.java:initialize(421)) - 42E95F5B438D456A81A3351F8C1E7B75: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:13,432 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:13,432 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[5F19CE8E2297420C955E344E29BD22B4:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [42E95F5B438D456A81A3351F8C1E7B75:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:13,433 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:13,433 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:13,433 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:13,433 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[5F19CE8E2297420C955E344E29BD22B4:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [42E95F5B438D456A81A3351F8C1E7B75:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:13,433 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:13,433 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 5F19CE8E2297420C955E344E29BD22B4 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:13,434 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 5F19CE8E2297420C955E344E29BD22B4
2016-05-18 00:58:13,434 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:13,434 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:13,434 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:13,434 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533093224_0001_1_00_000000_0
2016-05-18 00:58:13,435 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:13,435 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 5F19CE8E2297420C955E344E29BD22B4 initialized RecordReader from event
2016-05-18 00:58:13,438 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:13,439 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 5F19CE8E2297420C955E344E29BD22B4
2016-05-18 00:58:13,439 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_-1"], id: 42E95F5B438D456A81A3351F8C1E7B75
2016-05-18 00:58:13,439 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_-1"]
2016-05-18 00:58:13,439 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:13,440 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DAA31E5D794B4CC886049CBE7575C4F0, all 1 inputs ready in: 00:00:00.001
2016-05-18 00:58:13,444 INFO output.MROutput (MROutput.java:close(526)) - 42E95F5B438D456A81A3351F8C1E7B75 closed
2016-05-18 00:58:13,444 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533093224_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:13,445 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533093224_0001_1_00_000000_0
2016-05-18 00:58:13,445 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DAA31E5D794B4CC886049CBE7575C4F0, mem on close (mb), free: 284, total: 383, max: 672
2016-05-18 00:58:13,445 INFO output.MROutput (MROutput.java:close(526)) - 42E95F5B438D456A81A3351F8C1E7B75 closed
2016-05-18 00:58:13,445 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:13 Completed running task attempt: attempt_1463533093224_0001_1_00_000000_0
2016-05-18 00:58:13,446 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533093224_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:13,446 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533093224_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:13,447 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533093224_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533093440, Process_Duration=5, Process_End_Time=1463533093445, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:13,447 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533093224_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:13,448 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:13,448 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:13,448 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:13,448 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093224_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DAA31E5D794B4CC886049CBE7575C4F0, taskAttemptId=attempt_1463533093224_0001_1_00_000000_0, creationTime=1463533093408, allocationTime=1463533093410, startTime=1463533093415, finishTime=1463533093448, timeTaken=33, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:13,448 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533093224_0001_00_000001
2016-05-18 00:58:13,448 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533093224_0001_1_00_000000_0
2016-05-18 00:58:13,449 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093224_0001_1][Event:TASK_FINISHED]: vertexName=DAA31E5D794B4CC886049CBE7575C4F0, taskId=task_1463533093224_0001_1_00_000000, startTime=1463533093415, finishTime=1463533093449, timeTaken=34, status=SUCCEEDED, successfulAttemptID=attempt_1463533093224_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533093440, Process_Duration=5, Process_End_Time=1463533093445, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:13,450 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:13,450 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0]
2016-05-18 00:58:13,451 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093224_0001_1][Event:VERTEX_FINISHED]: vertexName=DAA31E5D794B4CC886049CBE7575C4F0, vertexId=vertex_1463533093224_0001_1_00, initRequestedTime=1463533093393, initedTime=1463533093403, startRequestedTime=1463533093395, startedTime=1463533093407, finishTime=1463533093450, timeTaken=43, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533093440, Process_Duration=5, Process_End_Time=1463533093445, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533093415, firstTasksToStart=[ task_1463533093224_0001_1_00_000000 ], lastTaskFinishTime=1463533093448, lastTasksToFinish=[ task_1463533093224_0001_1_00_000000 ], minTaskDuration=33, maxTaskDuration=33, avgTaskDuration=33.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533093224_0001_1_00_000000 ], longestDurationTasks=[ task_1463533093224_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:13,451 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:13,451 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093224_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533093224_0001_00_000001, stoppedTime=1463533093451, exitStatus=0
2016-05-18 00:58:13,451 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:13,451 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:13,451 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533093224_0001_1
2016-05-18 00:58:13,452 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093224_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533093224_0001_1
2016-05-18 00:58:13,452 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 42E95F5B438D456A81A3351F8C1E7B75 for vertex: vertex_1463533093224_0001_1_00 [DAA31E5D794B4CC886049CBE7575C4F0], outputName: 42E95F5B438D456A81A3351F8C1E7B75
2016-05-18 00:58:13,454 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533093224_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:13,454 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533093224_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:13,458 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:42E95F5B438D456A81A3351F8C1E7B75 of vertex/vertexGroup:DAA31E5D794B4CC886049CBE7575C4F0 isVertexGroupOutput:false
2016-05-18 00:58:13,458 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:13,459 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533093224_0001_1, queueSize=0
2016-05-18 00:58:13,460 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093224_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533093224_0001_1, startTime=1463533093393, finishTime=1463533093458, timeTaken=65, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=110, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533093440, Process_Duration=5, Process_End_Time=1463533093445, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:13,460 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533093224_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:13,460 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533093224_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:13,460 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533093224_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:13 Completed Dag: dag_1463533093224_0001_1
2016-05-18 00:58:13,460 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533093224_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:13,460 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533093224_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:13,461 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533093224_0001_00_000001, Exiting
2016-05-18 00:58:13,461 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533093224_0001_00_000001
2016-05-18 00:58:13,461 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533093224_0001_00_000001
2016-05-18 00:58:13,461 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533093224_0001_1_post
2016-05-18 00:58:13,461 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:13,461 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:13,461 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:13,461 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[03E175CC08FA43E48FD93993011AC086/ED07B0DBD2BD4CCB89F709B65F7D4EEB] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1, with id=dag_1463533093224_0001_1
2016-05-18 00:58:13,461 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533093224_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:13,462 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533093224_0001_00_000001 completed successfully
2016-05-18 00:58:13,462 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[03E175CC08FA43E48FD93993011AC086/ED07B0DBD2BD4CCB89F709B65F7D4EEB] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1, with id=dag_1463533093224_0001_1
2016-05-18 00:58:13,472 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...'_RESULTS.ALL_'field2'_-1, applicationId=application_1463533093224_0001
2016-05-18 00:58:13,474 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:13,474 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:13,474 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:13,474 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:13,474 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:13,474 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:58:13,474 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:58:13,475 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:13,476 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:13,476 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:13,476 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:13,476 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:13,477 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:13,477 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:13,477 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:13,478 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:13,481 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/03E175CC08FA43E48FD93993011AC086/ED07B0DBD2BD4CCB89F709B65F7D4EEB/.tez/application_1463533093224_0001
2016-05-18 00:58:13,481 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_-1/_temporary
2016-05-18 00:58:13,482 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.278
2016-05-18 00:58:13,550 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_'field2'_-1
2016-05-18 00:58:13,571 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:13,572 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:13,602 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-18 00:58:13,607 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-18 00:58:13,608 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:13,608 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:13,608 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:13,622 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:13,623 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field2'
2016-05-18 00:58:13,632 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:13,632 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:13,632 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field2'"]
2016-05-18 00:58:13,633 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:13,633 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:13,633 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:13,634 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...LTS.ALL_'field2'_'field2'
2016-05-18 00:58:13,634 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:13,647 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:13,651 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:13,652 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:13,652 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:13,661 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3312E8A21BE4709A8D975560DAD5F1D/D24E65DA70AC43199F8334D98B9113DB/.tez/application_1463533093652_0001 doesn't exist and is created
2016-05-18 00:58:13,675 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:13,675 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:13,676 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3312E8A21BE4709A8D975560DAD5F1D/D24E65DA70AC43199F8334D98B9113DB/.tez/application_1463533093652_0001_wd
2016-05-18 00:58:13,685 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533093652_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:13,686 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:13,686 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:13,688 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:13,688 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:13,688 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:13,688 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:13,690 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:13,691 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:13,692 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:13,692 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:13,693 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:13,693 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:13,693 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:13,693 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:13,693 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:13,694 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:13,694 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:13,695 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:13,695 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:13,697 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:13,698 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:13,698 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:13,698 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3312E8A21BE4709A8D975560DAD5F1D/D24E65DA70AC43199F8334D98B9113DB/.tez/application_1463533093652_0001_wd/localmode-log-dir
2016-05-18 00:58:13,699 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3312E8A21BE4709A8D975560DAD5F1D/D24E65DA70AC43199F8334D98B9113DB/.tez/application_1463533093652_0001_wd/localmode-log-dir/history.txt.appattempt_1463533093652_0001_000000, maxErrors=10
2016-05-18 00:58:13,699 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3312E8A21BE4709A8D975560DAD5F1D/D24E65DA70AC43199F8334D98B9113DB/.tez/application_1463533093652_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:13,699 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533093652_0001_000000, appSubmitTime=1463533093685, launchTime=1463533093685
2016-05-18 00:58:13,704 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:33157
2016-05-18 00:58:13,704 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:13,707 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533093652
2016-05-18 00:58:13,710 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533093652_0001_000000, startTime=1463533093710
2016-05-18 00:58:13,710 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:13,775 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:13,776 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:13,776 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...LTS.ALL_'field2'_'field2', applicationId=application_1463533093652_0001, dagName=[D3312E8A21BE4709A8D975560DAD5F1D/D24E65DA70AC43199F8334D98B9113DB] every.every/(1/1) ...LTS.ALL_'field2'_'field2'
2016-05-18 00:58:13,791 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [D3312E8A21BE4709A8D975560DAD5F1D/D24E65DA70AC43199F8334D98B9113DB] every.every/(1/1) ...LTS.ALL_'field2'_'field2'
2016-05-18 00:58:13,791 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:13,792 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533093652_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3312E8A21BE4709A8D975560DAD5F1D/D24E65DA70AC43199F8334D98B9113DB/.tez/application_1463533093652_0001_wd/localmode-log-dir/dag_1463533093652_0001_1.dot
2016-05-18 00:58:13,792 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3312E8A21BE4709A8D975560DAD5F1D/D24E65DA70AC43199F8334D98B9113DB/.tez/application_1463533093652_0001_wd/localmode-log-dir/dag_1463533093652_0001_1-tez-dag.pb.txt
2016-05-18 00:58:13,796 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533093652_0001_1
2016-05-18 00:58:13,796 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:13,796 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [D3312E8A21BE4709A8D975560DAD5F1D/D24E65DA70AC43199F8334D98B9113DB] every.every/(1/1) ...LTS.ALL_'field2'_'field2'
2016-05-18 00:58:13 Running Dag: dag_1463533093652_0001_1
2016-05-18 00:58:13,805 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093652_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533093652_0001_1, submitTime=1463533093791
2016-05-18 00:58:13,806 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3312E8A21BE4709A8D975560DAD5F1D/D24E65DA70AC43199F8334D98B9113DB/.tez/application_1463533093652_0001/tez-conf.pb" } size: 36913 timestamp: 1463533093000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:13,806 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4]
2016-05-18 00:58:13,806 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4]
2016-05-18 00:58:13,807 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:13,807 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:13,807 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093652_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533093652_0001_1, initTime=1463533093807
2016-05-18 00:58:13,808 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533093652_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:13,808 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:13,808 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093652_0001_1][Event:DAG_STARTED]: dagID=dag_1463533093652_0001_1, startTime=1463533093808
2016-05-18 00:58:13,808 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...LTS.ALL_'field2'_'field2', applicationId=application_1463533093652_0001, dagName=[D3312E8A21BE4709A8D975560DAD5F1D/D24E65DA70AC43199F8334D98B9113DB] every.every/(1/1) ...LTS.ALL_'field2'_'field2'
2016-05-18 00:58:13,808 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533093652_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:13,809 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: DD592E5872A447ABB31ECEA473D4DCB4 : {E6349803E00543FBA5091339B6827E71={InputName=E6349803E00543FBA5091339B6827E71}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:13,809 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: E6349803E00543FBA5091339B6827E71, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:13,809 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4]
2016-05-18 00:58:13,809 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4]
2016-05-18 00:58:13,809 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4]
2016-05-18 00:58:13,810 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4]
2016-05-18 00:58:13,810 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:13,810 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: E6349803E00543FBA5091339B6827E71 on vertex vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4]
2016-05-18 00:58:13,813 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input E6349803E00543FBA5091339B6827E71 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:13,814 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:13,814 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:13,815 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:13,815 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:13,815 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: E6349803E00543FBA5091339B6827E71 on vertex vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4]
2016-05-18 00:58:13,816 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:13,817 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {E6349803E00543FBA5091339B6827E71=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:13,819 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4] parallelism set to 1
2016-05-18 00:58:13,819 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:13,820 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4], numAdditionalOutputs=1
2016-05-18 00:58:13,820 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=EF11BFC7B58A464FAC0998362A864E76, vertex=vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:13,821 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for DD592E5872A447ABB31ECEA473D4DCB4:EF11BFC7B58A464FAC0998362A864E76 using oldmapred API
2016-05-18 00:58:13,821 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=EF11BFC7B58A464FAC0998362A864E76, vertexName=DD592E5872A447ABB31ECEA473D4DCB4, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:13,822 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093652_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DD592E5872A447ABB31ECEA473D4DCB4, vertexId=vertex_1463533093652_0001_1_00, initRequestedTime=1463533093809, initedTime=1463533093820, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:13,822 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4] with distanceFromRoot: 0
2016-05-18 00:58:13,823 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:13,823 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093652_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533093652_0001_1_00, startRequestedTime=1463533093810, startedTime=1463533093823
2016-05-18 00:58:13,823 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093652_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533093652_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:13,823 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:13,824 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in DD592E5872A447ABB31ECEA473D4DCB4
2016-05-18 00:58:13,824 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4]
2016-05-18 00:58:13,824 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093652_0001_1][Event:TASK_STARTED]: vertexName=DD592E5872A447ABB31ECEA473D4DCB4, taskId=task_1463533093652_0001_1_00_000000, scheduledTime=1463533093824, launchTime=1463533093824
2016-05-18 00:58:13,825 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4]
2016-05-18 00:58:13,825 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:13,827 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533093652_0001_00_000001, launchTime=1463533093827
2016-05-18 00:58:13,827 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533093652_0001_00_000001
2016-05-18 00:58:13,828 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533093652_0001_1_00_000000_0] started. Is using containerId: [container_1463533093652_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:13,828 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093652_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DD592E5872A447ABB31ECEA473D4DCB4, taskAttemptId=attempt_1463533093652_0001_1_00_000000_0, startTime=1463533093828, containerId=container_1463533093652_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:13,829 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533093652_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533093652_0001_1_00_000000_0
2016-05-18 00:58:13,829 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533093652_0001, with dag id: dag_1463533093652_0001_1
2016-05-18 00:58:13 Starting to run new task attempt: attempt_1463533093652_0001_1_00_000000_0
2016-05-18 00:58:13,832 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533093652_0001_1_00_000000_0
2016-05-18 00:58:13,832 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:13,832 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:13,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D3312E8A21BE4709A8D975560DAD5F1D/D24E65DA70AC43199F8334D98B9113DB] every.every/(1/1) ...LTS.ALL_'field2'_'field2', VertexName: DD592E5872A447ABB31ECEA473D4DCB4, VertexParallelism: 1, TaskAttemptID:attempt_1463533093652_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E6349803E00543FBA5091339B6827E71, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EF11BFC7B58A464FAC0998362A864E76, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:13,833 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:13,833 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533093652_0001_1_00_000000_0
2016-05-18 00:58:13,834 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533093652_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:13,834 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533093652_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:13,838 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:13,838 INFO input.MRInput (MRInput.java:initialize(440)) - E6349803E00543FBA5091339B6827E71 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:13,838 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: E6349803E00543FBA5091339B6827E71
2016-05-18 00:58:13,844 INFO output.MROutput (MROutput.java:initialize(421)) - EF11BFC7B58A464FAC0998362A864E76: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:13,845 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DD592E5872A447ABB31ECEA473D4DCB4, ordinal: 0
2016-05-18 00:58:13,845 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD592E5872A447ABB31ECEA473D4DCB4, mem on start (mb), free: 198, total: 383, max: 672
2016-05-18 00:58:13,845 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:13,845 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:13,845 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:13,845 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:13,845 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[E6349803E00543FBA5091339B6827E71:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [EF11BFC7B58A464FAC0998362A864E76:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:13,846 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:13,846 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:13,846 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:13,846 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[E6349803E00543FBA5091339B6827E71:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [EF11BFC7B58A464FAC0998362A864E76:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:13,846 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:13,846 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: E6349803E00543FBA5091339B6827E71
2016-05-18 00:58:13,846 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: E6349803E00543FBA5091339B6827E71 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:13,847 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:13,847 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:13,847 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:13,847 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533093652_0001_1_00_000000_0
2016-05-18 00:58:13,850 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:13,850 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:13,850 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - E6349803E00543FBA5091339B6827E71 initialized RecordReader from event
2016-05-18 00:58:13,850 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: E6349803E00543FBA5091339B6827E71
2016-05-18 00:58:13,850 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field2'"], id: EF11BFC7B58A464FAC0998362A864E76
2016-05-18 00:58:13,851 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field2'"]
2016-05-18 00:58:13,851 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:13,851 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DD592E5872A447ABB31ECEA473D4DCB4, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:13,855 INFO output.MROutput (MROutput.java:close(526)) - EF11BFC7B58A464FAC0998362A864E76 closed
2016-05-18 00:58:13,855 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533093652_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:13,856 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533093652_0001_1_00_000000_0
2016-05-18 00:58:13,856 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD592E5872A447ABB31ECEA473D4DCB4, mem on close (mb), free: 197, total: 383, max: 672
2016-05-18 00:58:13,856 INFO output.MROutput (MROutput.java:close(526)) - EF11BFC7B58A464FAC0998362A864E76 closed
2016-05-18 00:58:13,856 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:13 Completed running task attempt: attempt_1463533093652_0001_1_00_000000_0
2016-05-18 00:58:13,857 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533093652_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:13,857 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533093652_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:13,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533093652_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533093852, Process_Duration=4, Process_End_Time=1463533093856, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:13,859 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533093652_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:13,859 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:13,859 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093652_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DD592E5872A447ABB31ECEA473D4DCB4, taskAttemptId=attempt_1463533093652_0001_1_00_000000_0, creationTime=1463533093824, allocationTime=1463533093826, startTime=1463533093828, finishTime=1463533093859, timeTaken=31, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:13,859 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533093652_0001_1_00_000000_0
2016-05-18 00:58:13,859 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:13,860 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:13,860 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093652_0001_1][Event:TASK_FINISHED]: vertexName=DD592E5872A447ABB31ECEA473D4DCB4, taskId=task_1463533093652_0001_1_00_000000, startTime=1463533093828, finishTime=1463533093859, timeTaken=31, status=SUCCEEDED, successfulAttemptID=attempt_1463533093652_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533093852, Process_Duration=4, Process_End_Time=1463533093856, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:13,860 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533093652_0001_00_000001
2016-05-18 00:58:13,861 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:13,861 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4]
2016-05-18 00:58:13,862 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093652_0001_1][Event:VERTEX_FINISHED]: vertexName=DD592E5872A447ABB31ECEA473D4DCB4, vertexId=vertex_1463533093652_0001_1_00, initRequestedTime=1463533093809, initedTime=1463533093820, startRequestedTime=1463533093810, startedTime=1463533093823, finishTime=1463533093861, timeTaken=38, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533093852, Process_Duration=4, Process_End_Time=1463533093856, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533093828, firstTasksToStart=[ task_1463533093652_0001_1_00_000000 ], lastTaskFinishTime=1463533093859, lastTasksToFinish=[ task_1463533093652_0001_1_00_000000 ], minTaskDuration=31, maxTaskDuration=31, avgTaskDuration=31.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533093652_0001_1_00_000000 ], longestDurationTasks=[ task_1463533093652_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:13,862 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:13,862 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093652_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533093652_0001_00_000001, stoppedTime=1463533093862, exitStatus=0
2016-05-18 00:58:13,862 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:13,863 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:13,863 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533093652_0001_1
2016-05-18 00:58:13,863 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093652_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533093652_0001_1
2016-05-18 00:58:13,863 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: EF11BFC7B58A464FAC0998362A864E76 for vertex: vertex_1463533093652_0001_1_00 [DD592E5872A447ABB31ECEA473D4DCB4], outputName: EF11BFC7B58A464FAC0998362A864E76
2016-05-18 00:58:13,863 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533093652_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:13,864 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533093652_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:13,864 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533093652_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:13,864 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533093652_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:13,865 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533093652_0001_00_000001, Exiting
2016-05-18 00:58:13,865 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533093652_0001_00_000001
2016-05-18 00:58:13,865 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533093652_0001_00_000001
2016-05-18 00:58:13,867 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533093652_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:13,867 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533093652_0001_00_000001 completed successfully
2016-05-18 00:58:13,871 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:EF11BFC7B58A464FAC0998362A864E76 of vertex/vertexGroup:DD592E5872A447ABB31ECEA473D4DCB4 isVertexGroupOutput:false
2016-05-18 00:58:13,871 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:13,872 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533093652_0001_1, queueSize=0
2016-05-18 00:58:13,873 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533093652_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533093652_0001_1, startTime=1463533093808, finishTime=1463533093871, timeTaken=63, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=100, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533093852, Process_Duration=4, Process_End_Time=1463533093856, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:13,873 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533093652_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:13,873 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533093652_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:13 Completed Dag: dag_1463533093652_0001_1
2016-05-18 00:58:13,873 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533093652_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:13,874 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533093652_0001_1_post
2016-05-18 00:58:13,874 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:13,874 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:13,874 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:13,874 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[D3312E8A21BE4709A8D975560DAD5F1D/D24E65DA70AC43199F8334D98B9113DB] every.every/(1/1) ...LTS.ALL_'field2'_'field2', with id=dag_1463533093652_0001_1
2016-05-18 00:58:13,874 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[D3312E8A21BE4709A8D975560DAD5F1D/D24E65DA70AC43199F8334D98B9113DB] every.every/(1/1) ...LTS.ALL_'field2'_'field2', with id=dag_1463533093652_0001_1
2016-05-18 00:58:13,881 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...LTS.ALL_'field2'_'field2', applicationId=application_1463533093652_0001
2016-05-18 00:58:13,883 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:13,883 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:13,883 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:13,884 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:13,884 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:13,884 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:13,885 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:13,885 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:13,887 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:13,887 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:13,887 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:13,887 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:13,888 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:13,889 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:13,893 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3312E8A21BE4709A8D975560DAD5F1D/D24E65DA70AC43199F8334D98B9113DB/.tez/application_1463533093652_0001
2016-05-18 00:58:13,893 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field2'/_temporary
2016-05-18 00:58:13,894 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.261
2016-05-18 00:58:13,952 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:58:13,952 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:58:13,964 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_'field2'_'field2'
2016-05-18 00:58:13,987 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:13,988 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:14,024 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-18 00:58:14,029 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-18 00:58:14,029 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:14,030 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:14,030 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:14,045 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:14,046 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field'
2016-05-18 00:58:14,055 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:14,055 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:14,055 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field'"]
2016-05-18 00:58:14,055 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:14,055 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:14,056 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:14,056 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ULTS.ALL_'field2'_'field'
2016-05-18 00:58:14,056 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:14,071 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:14,075 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:14,075 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:14,075 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:14,085 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/0BEAD80430944E7FA58EAF6158BBE988/CC77C42D3CB843DAB3DF8C0B87B36895/.tez/application_1463533094075_0001 doesn't exist and is created
2016-05-18 00:58:14,098 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:14,099 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:14,099 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/0BEAD80430944E7FA58EAF6158BBE988/CC77C42D3CB843DAB3DF8C0B87B36895/.tez/application_1463533094075_0001_wd
2016-05-18 00:58:14,109 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533094075_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:14,109 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:14,109 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:14,111 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:14,111 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:14,111 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:14,111 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:14,112 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:14,113 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:14,113 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:14,113 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:14,114 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:14,114 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:14,114 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:14,114 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:14,114 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:14,114 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:14,115 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:14,115 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:14,115 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:14,116 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:14,117 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:14,117 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:14,118 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/0BEAD80430944E7FA58EAF6158BBE988/CC77C42D3CB843DAB3DF8C0B87B36895/.tez/application_1463533094075_0001_wd/localmode-log-dir
2016-05-18 00:58:14,118 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/0BEAD80430944E7FA58EAF6158BBE988/CC77C42D3CB843DAB3DF8C0B87B36895/.tez/application_1463533094075_0001_wd/localmode-log-dir/history.txt.appattempt_1463533094075_0001_000000, maxErrors=10
2016-05-18 00:58:14,118 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/0BEAD80430944E7FA58EAF6158BBE988/CC77C42D3CB843DAB3DF8C0B87B36895/.tez/application_1463533094075_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:14,118 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533094075_0001_000000, appSubmitTime=1463533094108, launchTime=1463533094109
2016-05-18 00:58:14,120 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:14,125 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:54327
2016-05-18 00:58:14,126 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533094075
2016-05-18 00:58:14,127 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533094075_0001_000000, startTime=1463533094127
2016-05-18 00:58:14,127 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:14,199 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:14,199 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:14,199 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ULTS.ALL_'field2'_'field', applicationId=application_1463533094075_0001, dagName=[0BEAD80430944E7FA58EAF6158BBE988/CC77C42D3CB843DAB3DF8C0B87B36895] every.every/(1/1) ...ULTS.ALL_'field2'_'field'
2016-05-18 00:58:14,215 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [0BEAD80430944E7FA58EAF6158BBE988/CC77C42D3CB843DAB3DF8C0B87B36895] every.every/(1/1) ...ULTS.ALL_'field2'_'field'
2016-05-18 00:58:14,215 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:14,216 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533094075_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/0BEAD80430944E7FA58EAF6158BBE988/CC77C42D3CB843DAB3DF8C0B87B36895/.tez/application_1463533094075_0001_wd/localmode-log-dir/dag_1463533094075_0001_1.dot
2016-05-18 00:58:14,216 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/0BEAD80430944E7FA58EAF6158BBE988/CC77C42D3CB843DAB3DF8C0B87B36895/.tez/application_1463533094075_0001_wd/localmode-log-dir/dag_1463533094075_0001_1-tez-dag.pb.txt
2016-05-18 00:58:14,219 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533094075_0001_1
2016-05-18 00:58:14,220 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:14,220 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [0BEAD80430944E7FA58EAF6158BBE988/CC77C42D3CB843DAB3DF8C0B87B36895] every.every/(1/1) ...ULTS.ALL_'field2'_'field'
2016-05-18 00:58:14 Running Dag: dag_1463533094075_0001_1
2016-05-18 00:58:14,229 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094075_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533094075_0001_1, submitTime=1463533094215
2016-05-18 00:58:14,230 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/0BEAD80430944E7FA58EAF6158BBE988/CC77C42D3CB843DAB3DF8C0B87B36895/.tez/application_1463533094075_0001/tez-conf.pb" } size: 36913 timestamp: 1463533094000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:14,231 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1]
2016-05-18 00:58:14,231 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1]
2016-05-18 00:58:14,231 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:14,231 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:14,232 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094075_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533094075_0001_1, initTime=1463533094232
2016-05-18 00:58:14,232 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533094075_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:14,232 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:14,232 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094075_0001_1][Event:DAG_STARTED]: dagID=dag_1463533094075_0001_1, startTime=1463533094232
2016-05-18 00:58:14,233 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533094075_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:14,233 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: A5D9AD93B65D4EACB87538B2E596FCD1 : {DAE56F880B7D4B9FAFFB13F55B6C81D7={InputName=DAE56F880B7D4B9FAFFB13F55B6C81D7}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:14,233 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: DAE56F880B7D4B9FAFFB13F55B6C81D7, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:14,233 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1]
2016-05-18 00:58:14,233 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1]
2016-05-18 00:58:14,233 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1]
2016-05-18 00:58:14,233 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1]
2016-05-18 00:58:14,234 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:14,234 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: DAE56F880B7D4B9FAFFB13F55B6C81D7 on vertex vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1]
2016-05-18 00:58:14,235 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ULTS.ALL_'field2'_'field', applicationId=application_1463533094075_0001, dagName=[0BEAD80430944E7FA58EAF6158BBE988/CC77C42D3CB843DAB3DF8C0B87B36895] every.every/(1/1) ...ULTS.ALL_'field2'_'field'
2016-05-18 00:58:14,236 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input DAE56F880B7D4B9FAFFB13F55B6C81D7 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:14,238 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:14,238 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:14,238 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:14,238 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:14,239 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: DAE56F880B7D4B9FAFFB13F55B6C81D7 on vertex vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1]
2016-05-18 00:58:14,241 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:14,242 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {DAE56F880B7D4B9FAFFB13F55B6C81D7=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:14,243 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1] parallelism set to 1
2016-05-18 00:58:14,243 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:14,244 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1], numAdditionalOutputs=1
2016-05-18 00:58:14,244 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=170AD7D3C5CC4B7C97F710A97B8638FC, vertex=vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:14,246 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for A5D9AD93B65D4EACB87538B2E596FCD1:170AD7D3C5CC4B7C97F710A97B8638FC using oldmapred API
2016-05-18 00:58:14,246 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=170AD7D3C5CC4B7C97F710A97B8638FC, vertexName=A5D9AD93B65D4EACB87538B2E596FCD1, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:14,246 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094075_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A5D9AD93B65D4EACB87538B2E596FCD1, vertexId=vertex_1463533094075_0001_1_00, initRequestedTime=1463533094233, initedTime=1463533094244, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:14,247 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1] with distanceFromRoot: 0
2016-05-18 00:58:14,247 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:14,247 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094075_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533094075_0001_1_00, startRequestedTime=1463533094234, startedTime=1463533094247
2016-05-18 00:58:14,247 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094075_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533094075_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:14,248 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:14,248 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in A5D9AD93B65D4EACB87538B2E596FCD1
2016-05-18 00:58:14,248 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1]
2016-05-18 00:58:14,249 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094075_0001_1][Event:TASK_STARTED]: vertexName=A5D9AD93B65D4EACB87538B2E596FCD1, taskId=task_1463533094075_0001_1_00_000000, scheduledTime=1463533094249, launchTime=1463533094249
2016-05-18 00:58:14,249 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1]
2016-05-18 00:58:14,250 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:14,252 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533094075_0001_00_000001, launchTime=1463533094252
2016-05-18 00:58:14,252 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533094075_0001_00_000001
2016-05-18 00:58:14,254 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533094075_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533094075_0001_1_00_000000_0
2016-05-18 00:58:14,254 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533094075_0001_1_00_000000_0] started. Is using containerId: [container_1463533094075_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:14,254 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094075_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A5D9AD93B65D4EACB87538B2E596FCD1, taskAttemptId=attempt_1463533094075_0001_1_00_000000_0, startTime=1463533094253, containerId=container_1463533094075_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:14,254 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533094075_0001, with dag id: dag_1463533094075_0001_1
2016-05-18 00:58:14 Starting to run new task attempt: attempt_1463533094075_0001_1_00_000000_0
2016-05-18 00:58:14,255 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533094075_0001_1_00_000000_0
2016-05-18 00:58:14,255 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:14,255 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:14,255 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0BEAD80430944E7FA58EAF6158BBE988/CC77C42D3CB843DAB3DF8C0B87B36895] every.every/(1/1) ...ULTS.ALL_'field2'_'field', VertexName: A5D9AD93B65D4EACB87538B2E596FCD1, VertexParallelism: 1, TaskAttemptID:attempt_1463533094075_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DAE56F880B7D4B9FAFFB13F55B6C81D7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=170AD7D3C5CC4B7C97F710A97B8638FC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:14,256 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:14,257 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533094075_0001_1_00_000000_0
2016-05-18 00:58:14,258 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533094075_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:14,259 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533094075_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:14,260 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:14,260 INFO input.MRInput (MRInput.java:initialize(440)) - DAE56F880B7D4B9FAFFB13F55B6C81D7 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:14,261 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: DAE56F880B7D4B9FAFFB13F55B6C81D7
2016-05-18 00:58:14,269 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A5D9AD93B65D4EACB87538B2E596FCD1, ordinal: 0
2016-05-18 00:58:14,270 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A5D9AD93B65D4EACB87538B2E596FCD1, mem on start (mb), free: 113, total: 383, max: 672
2016-05-18 00:58:14,270 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:14,270 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:14,270 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:14,271 INFO output.MROutput (MROutput.java:initialize(421)) - 170AD7D3C5CC4B7C97F710A97B8638FC: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:14,271 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:14,271 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[DAE56F880B7D4B9FAFFB13F55B6C81D7:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [170AD7D3C5CC4B7C97F710A97B8638FC:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:14,271 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:14,272 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:14,272 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:14,272 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[DAE56F880B7D4B9FAFFB13F55B6C81D7:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [170AD7D3C5CC4B7C97F710A97B8638FC:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:14,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:14,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: DAE56F880B7D4B9FAFFB13F55B6C81D7 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:14,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:14,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:14,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: DAE56F880B7D4B9FAFFB13F55B6C81D7
2016-05-18 00:58:14,274 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:14,274 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533094075_0001_1_00_000000_0
2016-05-18 00:58:14,277 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:14,277 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - DAE56F880B7D4B9FAFFB13F55B6C81D7 initialized RecordReader from event
2016-05-18 00:58:14,277 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:14,278 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: DAE56F880B7D4B9FAFFB13F55B6C81D7
2016-05-18 00:58:14,278 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field'"], id: 170AD7D3C5CC4B7C97F710A97B8638FC
2016-05-18 00:58:14,278 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field'"]
2016-05-18 00:58:14,279 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:14,279 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A5D9AD93B65D4EACB87538B2E596FCD1, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:14,282 INFO output.MROutput (MROutput.java:close(526)) - 170AD7D3C5CC4B7C97F710A97B8638FC closed
2016-05-18 00:58:14,282 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533094075_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:14,283 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533094075_0001_1_00_000000_0
2016-05-18 00:58:14,283 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A5D9AD93B65D4EACB87538B2E596FCD1, mem on close (mb), free: 111, total: 383, max: 672
2016-05-18 00:58:14,283 INFO output.MROutput (MROutput.java:close(526)) - 170AD7D3C5CC4B7C97F710A97B8638FC closed
2016-05-18 00:58:14,283 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:14 Completed running task attempt: attempt_1463533094075_0001_1_00_000000_0
2016-05-18 00:58:14,284 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533094075_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:14,284 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533094075_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:14,285 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533094075_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533094279, Process_Duration=4, Process_End_Time=1463533094283, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:14,286 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533094075_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:14,286 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:14,286 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094075_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A5D9AD93B65D4EACB87538B2E596FCD1, taskAttemptId=attempt_1463533094075_0001_1_00_000000_0, creationTime=1463533094249, allocationTime=1463533094251, startTime=1463533094253, finishTime=1463533094286, timeTaken=33, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:14,286 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:14,286 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:14,286 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533094075_0001_1_00_000000_0
2016-05-18 00:58:14,287 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094075_0001_1][Event:TASK_FINISHED]: vertexName=A5D9AD93B65D4EACB87538B2E596FCD1, taskId=task_1463533094075_0001_1_00_000000, startTime=1463533094253, finishTime=1463533094287, timeTaken=34, status=SUCCEEDED, successfulAttemptID=attempt_1463533094075_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533094279, Process_Duration=4, Process_End_Time=1463533094283, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:14,288 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:14,288 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1]
2016-05-18 00:58:14,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094075_0001_1][Event:VERTEX_FINISHED]: vertexName=A5D9AD93B65D4EACB87538B2E596FCD1, vertexId=vertex_1463533094075_0001_1_00, initRequestedTime=1463533094233, initedTime=1463533094244, startRequestedTime=1463533094234, startedTime=1463533094247, finishTime=1463533094288, timeTaken=41, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533094279, Process_Duration=4, Process_End_Time=1463533094283, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533094253, firstTasksToStart=[ task_1463533094075_0001_1_00_000000 ], lastTaskFinishTime=1463533094286, lastTasksToFinish=[ task_1463533094075_0001_1_00_000000 ], minTaskDuration=33, maxTaskDuration=33, avgTaskDuration=33.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533094075_0001_1_00_000000 ], longestDurationTasks=[ task_1463533094075_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:14,289 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:14,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094075_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533094075_0001_00_000001, stoppedTime=1463533094289, exitStatus=0
2016-05-18 00:58:14,289 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:14,289 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:14,290 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533094075_0001_1
2016-05-18 00:58:14,290 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094075_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533094075_0001_1
2016-05-18 00:58:14,290 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 170AD7D3C5CC4B7C97F710A97B8638FC for vertex: vertex_1463533094075_0001_1_00 [A5D9AD93B65D4EACB87538B2E596FCD1], outputName: 170AD7D3C5CC4B7C97F710A97B8638FC
2016-05-18 00:58:14,293 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533094075_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:14,293 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533094075_0001_00_000001
2016-05-18 00:58:14,293 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533094075_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:14,294 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533094075_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:14,294 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533094075_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:14,294 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533094075_0001_00_000001, Exiting
2016-05-18 00:58:14,294 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533094075_0001_00_000001
2016-05-18 00:58:14,294 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533094075_0001_00_000001
2016-05-18 00:58:14,295 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533094075_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:14,295 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533094075_0001_00_000001 completed successfully
2016-05-18 00:58:14,299 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:170AD7D3C5CC4B7C97F710A97B8638FC of vertex/vertexGroup:A5D9AD93B65D4EACB87538B2E596FCD1 isVertexGroupOutput:false
2016-05-18 00:58:14,299 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:14,300 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533094075_0001_1, queueSize=0
2016-05-18 00:58:14,301 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094075_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533094075_0001_1, startTime=1463533094232, finishTime=1463533094299, timeTaken=67, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=130, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533094279, Process_Duration=4, Process_End_Time=1463533094283, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:14,302 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533094075_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:14,302 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533094075_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:14 Completed Dag: dag_1463533094075_0001_1
2016-05-18 00:58:14,303 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533094075_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:14,303 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533094075_0001_1_post
2016-05-18 00:58:14,304 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:14,304 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:14,304 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:14,305 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[0BEAD80430944E7FA58EAF6158BBE988/CC77C42D3CB843DAB3DF8C0B87B36895] every.every/(1/1) ...ULTS.ALL_'field2'_'field', with id=dag_1463533094075_0001_1
2016-05-18 00:58:14,305 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[0BEAD80430944E7FA58EAF6158BBE988/CC77C42D3CB843DAB3DF8C0B87B36895] every.every/(1/1) ...ULTS.ALL_'field2'_'field', with id=dag_1463533094075_0001_1
2016-05-18 00:58:14,315 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ULTS.ALL_'field2'_'field', applicationId=application_1463533094075_0001
2016-05-18 00:58:14,317 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:14,317 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:14,317 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:14,317 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:14,317 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:14,318 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:14,318 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:14,319 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:14,319 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:14,319 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:14,320 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:14,320 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:14,321 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:14,321 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:14,325 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/0BEAD80430944E7FA58EAF6158BBE988/CC77C42D3CB843DAB3DF8C0B87B36895/.tez/application_1463533094075_0001
2016-05-18 00:58:14,325 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field'/_temporary
2016-05-18 00:58:14,326 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.270
2016-05-18 00:58:14,406 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_'field2'_'field'
2016-05-18 00:58:14,423 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:58:14,423 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:58:14,429 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:14,429 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:14,460 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-18 00:58:14,468 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-18 00:58:14,468 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:14,468 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:14,468 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:14,482 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:14,483 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_RESULTS
2016-05-18 00:58:14,491 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:14,492 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:14,492 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_RESULTS"]
2016-05-18 00:58:14,492 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:14,493 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:14,493 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:14,494 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ULTS.ALL_'field2'_RESULTS
2016-05-18 00:58:14,494 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:14,515 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:14,519 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:14,519 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:14,520 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:14,529 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/CE06B39334E74681B759DCE6E5D1091F/FE768F58304F48AFA020280A16B2C6D1/.tez/application_1463533094519_0001 doesn't exist and is created
2016-05-18 00:58:14,544 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:14,544 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:14,545 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/CE06B39334E74681B759DCE6E5D1091F/FE768F58304F48AFA020280A16B2C6D1/.tez/application_1463533094519_0001_wd
2016-05-18 00:58:14,555 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533094519_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:14,555 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:14,555 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:14,556 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:14,557 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:14,557 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:14,557 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:14,557 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:14,559 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:14,559 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:14,559 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:14,559 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:14,560 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:14,560 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:14,560 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:14,560 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:14,560 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:14,560 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:14,561 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:14,561 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:14,563 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:14,564 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:14,564 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:14,564 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/CE06B39334E74681B759DCE6E5D1091F/FE768F58304F48AFA020280A16B2C6D1/.tez/application_1463533094519_0001_wd/localmode-log-dir
2016-05-18 00:58:14,565 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/CE06B39334E74681B759DCE6E5D1091F/FE768F58304F48AFA020280A16B2C6D1/.tez/application_1463533094519_0001_wd/localmode-log-dir/history.txt.appattempt_1463533094519_0001_000000, maxErrors=10
2016-05-18 00:58:14,565 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/CE06B39334E74681B759DCE6E5D1091F/FE768F58304F48AFA020280A16B2C6D1/.tez/application_1463533094519_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:14,565 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533094519_0001_000000, appSubmitTime=1463533094554, launchTime=1463533094554
2016-05-18 00:58:14,568 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:14,572 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:37110
2016-05-18 00:58:14,573 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533094519
2016-05-18 00:58:14,574 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533094519_0001_000000, startTime=1463533094574
2016-05-18 00:58:14,575 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:14,644 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:14,644 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:14,645 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ULTS.ALL_'field2'_RESULTS, applicationId=application_1463533094519_0001, dagName=[CE06B39334E74681B759DCE6E5D1091F/FE768F58304F48AFA020280A16B2C6D1] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS
2016-05-18 00:58:14,660 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [CE06B39334E74681B759DCE6E5D1091F/FE768F58304F48AFA020280A16B2C6D1] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS
2016-05-18 00:58:14,661 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:14,661 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533094519_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/CE06B39334E74681B759DCE6E5D1091F/FE768F58304F48AFA020280A16B2C6D1/.tez/application_1463533094519_0001_wd/localmode-log-dir/dag_1463533094519_0001_1.dot
2016-05-18 00:58:14,662 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/CE06B39334E74681B759DCE6E5D1091F/FE768F58304F48AFA020280A16B2C6D1/.tez/application_1463533094519_0001_wd/localmode-log-dir/dag_1463533094519_0001_1-tez-dag.pb.txt
2016-05-18 00:58:14,665 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533094519_0001_1
2016-05-18 00:58:14,665 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:14,666 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [CE06B39334E74681B759DCE6E5D1091F/FE768F58304F48AFA020280A16B2C6D1] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS
2016-05-18 00:58:14 Running Dag: dag_1463533094519_0001_1
2016-05-18 00:58:14,674 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094519_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533094519_0001_1, submitTime=1463533094660
2016-05-18 00:58:14,675 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/CE06B39334E74681B759DCE6E5D1091F/FE768F58304F48AFA020280A16B2C6D1/.tez/application_1463533094519_0001/tez-conf.pb" } size: 36913 timestamp: 1463533094000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:14,676 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9]
2016-05-18 00:58:14,676 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9]
2016-05-18 00:58:14,676 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:14,676 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:14,676 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094519_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533094519_0001_1, initTime=1463533094676
2016-05-18 00:58:14,676 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533094519_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:14,677 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:14,677 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094519_0001_1][Event:DAG_STARTED]: dagID=dag_1463533094519_0001_1, startTime=1463533094677
2016-05-18 00:58:14,677 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533094519_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:14,677 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ULTS.ALL_'field2'_RESULTS, applicationId=application_1463533094519_0001, dagName=[CE06B39334E74681B759DCE6E5D1091F/FE768F58304F48AFA020280A16B2C6D1] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS
2016-05-18 00:58:14,677 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: BF6A3BC6FC5B421F9E6B499C91AAE2F9 : {0D9765A65B6549E98F84D9C022643202={InputName=0D9765A65B6549E98F84D9C022643202}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:14,678 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 0D9765A65B6549E98F84D9C022643202, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:14,678 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9]
2016-05-18 00:58:14,678 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9]
2016-05-18 00:58:14,678 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9]
2016-05-18 00:58:14,678 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9]
2016-05-18 00:58:14,679 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:14,679 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 0D9765A65B6549E98F84D9C022643202 on vertex vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9]
2016-05-18 00:58:14,681 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 0D9765A65B6549E98F84D9C022643202 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:14,683 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:14,683 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:14,684 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:14,684 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:14,684 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 0D9765A65B6549E98F84D9C022643202 on vertex vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9]
2016-05-18 00:58:14,686 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:14,686 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {0D9765A65B6549E98F84D9C022643202=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:14,686 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9] parallelism set to 1
2016-05-18 00:58:14,687 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:14,687 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9], numAdditionalOutputs=1
2016-05-18 00:58:14,687 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=B5703AC6691B4D078C83582DEC53F5C8, vertex=vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:14,688 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for BF6A3BC6FC5B421F9E6B499C91AAE2F9:B5703AC6691B4D078C83582DEC53F5C8 using oldmapred API
2016-05-18 00:58:14,688 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=B5703AC6691B4D078C83582DEC53F5C8, vertexName=BF6A3BC6FC5B421F9E6B499C91AAE2F9, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:14,689 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094519_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BF6A3BC6FC5B421F9E6B499C91AAE2F9, vertexId=vertex_1463533094519_0001_1_00, initRequestedTime=1463533094677, initedTime=1463533094687, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:14,689 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9] with distanceFromRoot: 0
2016-05-18 00:58:14,689 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:14,689 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094519_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533094519_0001_1_00, startRequestedTime=1463533094679, startedTime=1463533094689
2016-05-18 00:58:14,689 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094519_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533094519_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:14,690 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:14,690 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in BF6A3BC6FC5B421F9E6B499C91AAE2F9
2016-05-18 00:58:14,690 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9]
2016-05-18 00:58:14,691 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094519_0001_1][Event:TASK_STARTED]: vertexName=BF6A3BC6FC5B421F9E6B499C91AAE2F9, taskId=task_1463533094519_0001_1_00_000000, scheduledTime=1463533094691, launchTime=1463533094691
2016-05-18 00:58:14,691 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9]
2016-05-18 00:58:14,692 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:14,694 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533094519_0001_00_000001, launchTime=1463533094693
2016-05-18 00:58:14,694 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533094519_0001_00_000001
2016-05-18 00:58:14,695 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533094519_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533094519_0001_1_00_000000_0
2016-05-18 00:58:14,695 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533094519_0001_1_00_000000_0] started. Is using containerId: [container_1463533094519_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:14,695 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094519_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BF6A3BC6FC5B421F9E6B499C91AAE2F9, taskAttemptId=attempt_1463533094519_0001_1_00_000000_0, startTime=1463533094695, containerId=container_1463533094519_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:14,696 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533094519_0001, with dag id: dag_1463533094519_0001_1
2016-05-18 00:58:14 Starting to run new task attempt: attempt_1463533094519_0001_1_00_000000_0
2016-05-18 00:58:14,699 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533094519_0001_1_00_000000_0
2016-05-18 00:58:14,699 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:14,699 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:14,700 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [CE06B39334E74681B759DCE6E5D1091F/FE768F58304F48AFA020280A16B2C6D1] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS, VertexName: BF6A3BC6FC5B421F9E6B499C91AAE2F9, VertexParallelism: 1, TaskAttemptID:attempt_1463533094519_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0D9765A65B6549E98F84D9C022643202, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B5703AC6691B4D078C83582DEC53F5C8, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:14,700 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:14,701 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533094519_0001_1_00_000000_0
2016-05-18 00:58:14,701 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533094519_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:14,702 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533094519_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:14,707 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:14,708 INFO input.MRInput (MRInput.java:initialize(440)) - 0D9765A65B6549E98F84D9C022643202 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:14,708 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 0D9765A65B6549E98F84D9C022643202
2016-05-18 00:58:14,713 INFO output.MROutput (MROutput.java:initialize(421)) - B5703AC6691B4D078C83582DEC53F5C8: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:14,717 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BF6A3BC6FC5B421F9E6B499C91AAE2F9, ordinal: 0
2016-05-18 00:58:14,717 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BF6A3BC6FC5B421F9E6B499C91AAE2F9, mem on start (mb), free: 228, total: 384, max: 672
2016-05-18 00:58:14,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:14,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:14,718 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:14,718 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:14,718 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[0D9765A65B6549E98F84D9C022643202:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [B5703AC6691B4D078C83582DEC53F5C8:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:14,718 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:14,719 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:14,719 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:14,719 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[0D9765A65B6549E98F84D9C022643202:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [B5703AC6691B4D078C83582DEC53F5C8:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:14,719 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:14,720 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 0D9765A65B6549E98F84D9C022643202 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:14,720 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 0D9765A65B6549E98F84D9C022643202
2016-05-18 00:58:14,720 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:14,720 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:14,721 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:14,721 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533094519_0001_1_00_000000_0
2016-05-18 00:58:14,723 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:14,724 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 0D9765A65B6549E98F84D9C022643202 initialized RecordReader from event
2016-05-18 00:58:14,724 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:14,725 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 0D9765A65B6549E98F84D9C022643202
2016-05-18 00:58:14,725 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_RESULTS"], id: B5703AC6691B4D078C83582DEC53F5C8
2016-05-18 00:58:14,725 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_RESULTS"]
2016-05-18 00:58:14,726 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:14,726 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BF6A3BC6FC5B421F9E6B499C91AAE2F9, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:14,730 INFO output.MROutput (MROutput.java:close(526)) - B5703AC6691B4D078C83582DEC53F5C8 closed
2016-05-18 00:58:14,731 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533094519_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:14,731 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533094519_0001_1_00_000000_0
2016-05-18 00:58:14,732 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BF6A3BC6FC5B421F9E6B499C91AAE2F9, mem on close (mb), free: 226, total: 384, max: 672
2016-05-18 00:58:14,732 INFO output.MROutput (MROutput.java:close(526)) - B5703AC6691B4D078C83582DEC53F5C8 closed
2016-05-18 00:58:14,732 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:14 Completed running task attempt: attempt_1463533094519_0001_1_00_000000_0
2016-05-18 00:58:14,733 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533094519_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:14,733 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533094519_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:14,733 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533094519_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533094727, Process_Duration=4, Process_End_Time=1463533094731, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:14,734 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533094519_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:14,734 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:14,734 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:14,734 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:14,735 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094519_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BF6A3BC6FC5B421F9E6B499C91AAE2F9, taskAttemptId=attempt_1463533094519_0001_1_00_000000_0, creationTime=1463533094691, allocationTime=1463533094693, startTime=1463533094695, finishTime=1463533094735, timeTaken=40, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:14,735 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533094519_0001_1_00_000000_0
2016-05-18 00:58:14,735 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094519_0001_1][Event:TASK_FINISHED]: vertexName=BF6A3BC6FC5B421F9E6B499C91AAE2F9, taskId=task_1463533094519_0001_1_00_000000, startTime=1463533094695, finishTime=1463533094735, timeTaken=40, status=SUCCEEDED, successfulAttemptID=attempt_1463533094519_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533094727, Process_Duration=4, Process_End_Time=1463533094731, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:14,735 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:14,736 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9]
2016-05-18 00:58:14,736 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094519_0001_1][Event:VERTEX_FINISHED]: vertexName=BF6A3BC6FC5B421F9E6B499C91AAE2F9, vertexId=vertex_1463533094519_0001_1_00, initRequestedTime=1463533094677, initedTime=1463533094687, startRequestedTime=1463533094679, startedTime=1463533094689, finishTime=1463533094736, timeTaken=47, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533094727, Process_Duration=4, Process_End_Time=1463533094731, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533094695, firstTasksToStart=[ task_1463533094519_0001_1_00_000000 ], lastTaskFinishTime=1463533094735, lastTasksToFinish=[ task_1463533094519_0001_1_00_000000 ], minTaskDuration=40, maxTaskDuration=40, avgTaskDuration=40.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533094519_0001_1_00_000000 ], longestDurationTasks=[ task_1463533094519_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:14,737 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:14,737 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:14,737 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:14,737 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533094519_0001_1
2016-05-18 00:58:14,737 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094519_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533094519_0001_1
2016-05-18 00:58:14,737 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533094519_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:14,737 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: B5703AC6691B4D078C83582DEC53F5C8 for vertex: vertex_1463533094519_0001_1_00 [BF6A3BC6FC5B421F9E6B499C91AAE2F9], outputName: B5703AC6691B4D078C83582DEC53F5C8
2016-05-18 00:58:14,738 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094519_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533094519_0001_00_000001, stoppedTime=1463533094738, exitStatus=0
2016-05-18 00:58:14,738 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533094519_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:14,739 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533094519_0001_00_000001
2016-05-18 00:58:14,740 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533094519_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:14,740 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533094519_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:14,740 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533094519_0001_00_000001, Exiting
2016-05-18 00:58:14,740 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533094519_0001_00_000001
2016-05-18 00:58:14,740 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533094519_0001_00_000001
2016-05-18 00:58:14,742 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533094519_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:14,742 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533094519_0001_00_000001 completed successfully
2016-05-18 00:58:14,745 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:B5703AC6691B4D078C83582DEC53F5C8 of vertex/vertexGroup:BF6A3BC6FC5B421F9E6B499C91AAE2F9 isVertexGroupOutput:false
2016-05-18 00:58:14,745 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:14,747 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533094519_0001_1, queueSize=0
2016-05-18 00:58:14,747 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094519_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533094519_0001_1, startTime=1463533094677, finishTime=1463533094745, timeTaken=68, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=140, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533094727, Process_Duration=4, Process_End_Time=1463533094731, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:14,747 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533094519_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:14,747 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533094519_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:14 Completed Dag: dag_1463533094519_0001_1
2016-05-18 00:58:14,748 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533094519_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:14,748 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533094519_0001_1_post
2016-05-18 00:58:14,748 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:14,748 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:14,748 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:14,748 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[CE06B39334E74681B759DCE6E5D1091F/FE768F58304F48AFA020280A16B2C6D1] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS, with id=dag_1463533094519_0001_1
2016-05-18 00:58:14,748 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[CE06B39334E74681B759DCE6E5D1091F/FE768F58304F48AFA020280A16B2C6D1] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS, with id=dag_1463533094519_0001_1
2016-05-18 00:58:14,759 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ULTS.ALL_'field2'_RESULTS, applicationId=application_1463533094519_0001
2016-05-18 00:58:14,760 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:14,760 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:14,760 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:14,760 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:14,760 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:14,761 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:14,761 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:14,761 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:14,761 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:14,761 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:14,762 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:14,762 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:14,762 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:14,763 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:14,765 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/CE06B39334E74681B759DCE6E5D1091F/FE768F58304F48AFA020280A16B2C6D1/.tez/application_1463533094519_0001
2016-05-18 00:58:14,766 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_RESULTS/_temporary
2016-05-18 00:58:14,767 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.275
2016-05-18 00:58:14,840 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_'field2'_RESULTS
2016-05-18 00:58:14,852 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:58:14,852 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:58:14,865 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:14,866 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:14,897 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-18 00:58:14,904 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-18 00:58:14,905 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:14,905 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:14,906 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:14,921 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:14,922 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_ALL
2016-05-18 00:58:14,931 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:14,932 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:14,932 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_ALL"]
2016-05-18 00:58:14,932 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:14,932 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:14,932 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:14,933 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ..._RESULTS.ALL_'field2'_ALL
2016-05-18 00:58:14,933 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:14,950 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:14,955 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:14,956 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:14,956 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:14,966 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7F601743A1604F7091A6DB992C950161/D9C90783261444F69FE35889E11406EC/.tez/application_1463533094956_0001 doesn't exist and is created
2016-05-18 00:58:14,983 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:14,983 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:14,984 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7F601743A1604F7091A6DB992C950161/D9C90783261444F69FE35889E11406EC/.tez/application_1463533094956_0001_wd
2016-05-18 00:58:14,996 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533094956_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:14,997 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:14,997 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:14,999 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:14,999 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:14,999 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:14,999 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:15,000 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:15,001 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:15,001 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:15,001 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:15,002 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:15,002 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:15,002 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:15,002 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:15,002 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:15,002 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:15,003 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:15,003 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:15,003 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:15,005 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:15,006 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:15,006 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:15,006 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7F601743A1604F7091A6DB992C950161/D9C90783261444F69FE35889E11406EC/.tez/application_1463533094956_0001_wd/localmode-log-dir
2016-05-18 00:58:15,006 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7F601743A1604F7091A6DB992C950161/D9C90783261444F69FE35889E11406EC/.tez/application_1463533094956_0001_wd/localmode-log-dir/history.txt.appattempt_1463533094956_0001_000000, maxErrors=10
2016-05-18 00:58:15,007 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7F601743A1604F7091A6DB992C950161/D9C90783261444F69FE35889E11406EC/.tez/application_1463533094956_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:15,007 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533094956_0001_000000, appSubmitTime=1463533094996, launchTime=1463533094996
2016-05-18 00:58:15,017 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:15,017 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:52108
2016-05-18 00:58:15,019 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533094956
2016-05-18 00:58:15,021 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533094956_0001_000000, startTime=1463533095021
2016-05-18 00:58:15,021 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:15,084 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:15,084 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:15,084 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ..._RESULTS.ALL_'field2'_ALL, applicationId=application_1463533094956_0001, dagName=[7F601743A1604F7091A6DB992C950161/D9C90783261444F69FE35889E11406EC] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL
2016-05-18 00:58:15,109 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [7F601743A1604F7091A6DB992C950161/D9C90783261444F69FE35889E11406EC] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL
2016-05-18 00:58:15,109 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:15,110 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533094956_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7F601743A1604F7091A6DB992C950161/D9C90783261444F69FE35889E11406EC/.tez/application_1463533094956_0001_wd/localmode-log-dir/dag_1463533094956_0001_1.dot
2016-05-18 00:58:15,110 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7F601743A1604F7091A6DB992C950161/D9C90783261444F69FE35889E11406EC/.tez/application_1463533094956_0001_wd/localmode-log-dir/dag_1463533094956_0001_1-tez-dag.pb.txt
2016-05-18 00:58:15,114 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533094956_0001_1
2016-05-18 00:58:15,114 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:15,114 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [7F601743A1604F7091A6DB992C950161/D9C90783261444F69FE35889E11406EC] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL
2016-05-18 00:58:15 Running Dag: dag_1463533094956_0001_1
2016-05-18 00:58:15,124 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094956_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533094956_0001_1, submitTime=1463533095109
2016-05-18 00:58:15,125 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7F601743A1604F7091A6DB992C950161/D9C90783261444F69FE35889E11406EC/.tez/application_1463533094956_0001/tez-conf.pb" } size: 36913 timestamp: 1463533094000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:15,126 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA]
2016-05-18 00:58:15,126 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA]
2016-05-18 00:58:15,126 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:15,127 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:15,127 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094956_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533094956_0001_1, initTime=1463533095127
2016-05-18 00:58:15,127 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533094956_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:15,128 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:15,128 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094956_0001_1][Event:DAG_STARTED]: dagID=dag_1463533094956_0001_1, startTime=1463533095128
2016-05-18 00:58:15,128 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533094956_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:15,128 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: D06B149210E94DFFADAD562FEFA1FBBA : {AC782D7FB7DD409E996E3588C2311F92={InputName=AC782D7FB7DD409E996E3588C2311F92}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:15,128 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: AC782D7FB7DD409E996E3588C2311F92, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:15,129 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA]
2016-05-18 00:58:15,129 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA]
2016-05-18 00:58:15,129 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA]
2016-05-18 00:58:15,129 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA]
2016-05-18 00:58:15,130 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:15,132 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: AC782D7FB7DD409E996E3588C2311F92 on vertex vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA]
2016-05-18 00:58:15,133 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ..._RESULTS.ALL_'field2'_ALL, applicationId=application_1463533094956_0001, dagName=[7F601743A1604F7091A6DB992C950161/D9C90783261444F69FE35889E11406EC] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL
2016-05-18 00:58:15,134 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input AC782D7FB7DD409E996E3588C2311F92 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:15,135 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:15,135 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:15,135 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:15,136 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:15,136 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: AC782D7FB7DD409E996E3588C2311F92 on vertex vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA]
2016-05-18 00:58:15,137 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:15,137 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {AC782D7FB7DD409E996E3588C2311F92=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:15,139 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA] parallelism set to 1
2016-05-18 00:58:15,139 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:15,140 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA], numAdditionalOutputs=1
2016-05-18 00:58:15,140 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=220452622B42406A812E9AD3B6AD8A50, vertex=vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:15,141 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D06B149210E94DFFADAD562FEFA1FBBA:220452622B42406A812E9AD3B6AD8A50 using oldmapred API
2016-05-18 00:58:15,142 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=220452622B42406A812E9AD3B6AD8A50, vertexName=D06B149210E94DFFADAD562FEFA1FBBA, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:15,142 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094956_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D06B149210E94DFFADAD562FEFA1FBBA, vertexId=vertex_1463533094956_0001_1_00, initRequestedTime=1463533095128, initedTime=1463533095140, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:15,142 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA] with distanceFromRoot: 0
2016-05-18 00:58:15,143 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:15,143 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094956_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533094956_0001_1_00, startRequestedTime=1463533095130, startedTime=1463533095143
2016-05-18 00:58:15,143 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094956_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533094956_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:15,143 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:15,144 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in D06B149210E94DFFADAD562FEFA1FBBA
2016-05-18 00:58:15,144 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA]
2016-05-18 00:58:15,144 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094956_0001_1][Event:TASK_STARTED]: vertexName=D06B149210E94DFFADAD562FEFA1FBBA, taskId=task_1463533094956_0001_1_00_000000, scheduledTime=1463533095144, launchTime=1463533095144
2016-05-18 00:58:15,145 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA]
2016-05-18 00:58:15,145 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:15,147 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533094956_0001_00_000001, launchTime=1463533095147
2016-05-18 00:58:15,148 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533094956_0001_00_000001
2016-05-18 00:58:15,151 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533094956_0001_1_00_000000_0] started. Is using containerId: [container_1463533094956_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:15,151 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094956_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D06B149210E94DFFADAD562FEFA1FBBA, taskAttemptId=attempt_1463533094956_0001_1_00_000000_0, startTime=1463533095151, containerId=container_1463533094956_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:15,152 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533094956_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533094956_0001_1_00_000000_0
2016-05-18 00:58:15 Starting to run new task attempt: attempt_1463533094956_0001_1_00_000000_0
2016-05-18 00:58:15,152 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533094956_0001_1_00_000000_0
2016-05-18 00:58:15,152 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:15,153 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:15,153 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7F601743A1604F7091A6DB992C950161/D9C90783261444F69FE35889E11406EC] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL, VertexName: D06B149210E94DFFADAD562FEFA1FBBA, VertexParallelism: 1, TaskAttemptID:attempt_1463533094956_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AC782D7FB7DD409E996E3588C2311F92, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=220452622B42406A812E9AD3B6AD8A50, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:15,153 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:15,153 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533094956_0001_1_00_000000_0
2016-05-18 00:58:15,154 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533094956_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:15,155 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533094956_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:15,157 INFO input.MRInput (MRInput.java:initialize(440)) - AC782D7FB7DD409E996E3588C2311F92 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:15,157 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: AC782D7FB7DD409E996E3588C2311F92
2016-05-18 00:58:15,161 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:15,170 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D06B149210E94DFFADAD562FEFA1FBBA, ordinal: 0
2016-05-18 00:58:15,170 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D06B149210E94DFFADAD562FEFA1FBBA, mem on start (mb), free: 141, total: 384, max: 672
2016-05-18 00:58:15,170 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:15,170 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:15,171 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:15,170 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533094956_0001, with dag id: dag_1463533094956_0001_1
2016-05-18 00:58:15,171 INFO output.MROutput (MROutput.java:initialize(421)) - 220452622B42406A812E9AD3B6AD8A50: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:15,172 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:15,173 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[AC782D7FB7DD409E996E3588C2311F92:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [220452622B42406A812E9AD3B6AD8A50:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:58:15,174 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:15,174 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:15,174 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:15,175 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[AC782D7FB7DD409E996E3588C2311F92:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [220452622B42406A812E9AD3B6AD8A50:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:58:15,175 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:15,175 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: AC782D7FB7DD409E996E3588C2311F92
2016-05-18 00:58:15,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: AC782D7FB7DD409E996E3588C2311F92 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:15,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:15,177 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:15,177 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:15,177 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533094956_0001_1_00_000000_0
2016-05-18 00:58:15,181 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:15,181 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: AC782D7FB7DD409E996E3588C2311F92
2016-05-18 00:58:15,181 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_ALL"], id: 220452622B42406A812E9AD3B6AD8A50
2016-05-18 00:58:15,182 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_ALL"]
2016-05-18 00:58:15,182 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:15,184 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:15,184 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - AC782D7FB7DD409E996E3588C2311F92 initialized RecordReader from event
2016-05-18 00:58:15,184 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D06B149210E94DFFADAD562FEFA1FBBA, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:15,188 INFO output.MROutput (MROutput.java:close(526)) - 220452622B42406A812E9AD3B6AD8A50 closed
2016-05-18 00:58:15,188 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533094956_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:15,189 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533094956_0001_1_00_000000_0
2016-05-18 00:58:15,189 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D06B149210E94DFFADAD562FEFA1FBBA, mem on close (mb), free: 139, total: 384, max: 672
2016-05-18 00:58:15,189 INFO output.MROutput (MROutput.java:close(526)) - 220452622B42406A812E9AD3B6AD8A50 closed
2016-05-18 00:58:15,189 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:15 Completed running task attempt: attempt_1463533094956_0001_1_00_000000_0
2016-05-18 00:58:15,191 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533094956_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:15,191 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533094956_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:15,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533094956_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533095185, Process_Duration=4, Process_End_Time=1463533095189, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:15,195 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533094956_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:15,195 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:15,195 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:15,195 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:15,195 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533094956_0001_00_000001
2016-05-18 00:58:15,195 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094956_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D06B149210E94DFFADAD562FEFA1FBBA, taskAttemptId=attempt_1463533094956_0001_1_00_000000_0, creationTime=1463533095145, allocationTime=1463533095147, startTime=1463533095151, finishTime=1463533095195, timeTaken=44, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:15,196 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533094956_0001_1_00_000000_0
2016-05-18 00:58:15,197 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094956_0001_1][Event:TASK_FINISHED]: vertexName=D06B149210E94DFFADAD562FEFA1FBBA, taskId=task_1463533094956_0001_1_00_000000, startTime=1463533095151, finishTime=1463533095196, timeTaken=45, status=SUCCEEDED, successfulAttemptID=attempt_1463533094956_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533095185, Process_Duration=4, Process_End_Time=1463533095189, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:15,197 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:15,197 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA]
2016-05-18 00:58:15,198 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094956_0001_1][Event:VERTEX_FINISHED]: vertexName=D06B149210E94DFFADAD562FEFA1FBBA, vertexId=vertex_1463533094956_0001_1_00, initRequestedTime=1463533095128, initedTime=1463533095140, startRequestedTime=1463533095130, startedTime=1463533095143, finishTime=1463533095197, timeTaken=54, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533095185, Process_Duration=4, Process_End_Time=1463533095189, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533095151, firstTasksToStart=[ task_1463533094956_0001_1_00_000000 ], lastTaskFinishTime=1463533095195, lastTasksToFinish=[ task_1463533094956_0001_1_00_000000 ], minTaskDuration=44, maxTaskDuration=44, avgTaskDuration=44.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533094956_0001_1_00_000000 ], longestDurationTasks=[ task_1463533094956_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:15,198 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:15,199 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:15,199 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:15,199 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533094956_0001_1
2016-05-18 00:58:15,199 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094956_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533094956_0001_1
2016-05-18 00:58:15,203 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 220452622B42406A812E9AD3B6AD8A50 for vertex: vertex_1463533094956_0001_1_00 [D06B149210E94DFFADAD562FEFA1FBBA], outputName: 220452622B42406A812E9AD3B6AD8A50
2016-05-18 00:58:15,205 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533094956_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:15,206 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094956_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533094956_0001_00_000001, stoppedTime=1463533095206, exitStatus=0
2016-05-18 00:58:15,206 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533094956_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:15,207 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533094956_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:15,208 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533094956_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:15,208 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533094956_0001_00_000001, Exiting
2016-05-18 00:58:15,208 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533094956_0001_00_000001
2016-05-18 00:58:15,208 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533094956_0001_00_000001
2016-05-18 00:58:15,211 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:220452622B42406A812E9AD3B6AD8A50 of vertex/vertexGroup:D06B149210E94DFFADAD562FEFA1FBBA isVertexGroupOutput:false
2016-05-18 00:58:15,211 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:15,212 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533094956_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:15,212 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533094956_0001_1, queueSize=0
2016-05-18 00:58:15,213 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533094956_0001_00_000001 completed successfully
2016-05-18 00:58:15,213 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533094956_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533094956_0001_1, startTime=1463533095128, finishTime=1463533095211, timeTaken=83, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=240, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533095185, Process_Duration=4, Process_End_Time=1463533095189, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:15,214 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533094956_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:15,214 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533094956_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:15 Completed Dag: dag_1463533094956_0001_1
2016-05-18 00:58:15,215 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533094956_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:15,215 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533094956_0001_1_post
2016-05-18 00:58:15,215 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:15,215 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:15,215 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:15,215 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[7F601743A1604F7091A6DB992C950161/D9C90783261444F69FE35889E11406EC] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL, with id=dag_1463533094956_0001_1
2016-05-18 00:58:15,215 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[7F601743A1604F7091A6DB992C950161/D9C90783261444F69FE35889E11406EC] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL, with id=dag_1463533094956_0001_1
2016-05-18 00:58:15,225 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ..._RESULTS.ALL_'field2'_ALL, applicationId=application_1463533094956_0001
2016-05-18 00:58:15,227 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:15,227 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:15,227 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:15,227 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:15,227 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:15,228 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:15,228 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:15,230 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:15,230 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:15,230 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:15,231 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:15,231 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:15,231 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:15,232 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:15,236 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7F601743A1604F7091A6DB992C950161/D9C90783261444F69FE35889E11406EC/.tez/application_1463533094956_0001
2016-05-18 00:58:15,237 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_ALL/_temporary
2016-05-18 00:58:15,237 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.306
2016-05-18 00:58:15,320 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_'field2'_ALL
2016-05-18 00:58:15,340 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:58:15,340 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:58:15,344 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:15,344 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:15,375 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-05-18 00:58:15,381 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-18 00:58:15,382 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:15,383 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:58:15,383 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:58:15,411 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 00:58:15,411 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_UNKNOWN_-1
2016-05-18 00:58:15,423 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 00:58:15,423 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:15,424 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_UNKNOWN_-1"]
2016-05-18 00:58:15,424 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 00:58:15,424 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 00:58:15,424 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 00:58:15,424 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...d'_RESULTS.ALL_UNKNOWN_-1
2016-05-18 00:58:15,425 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:58:15,440 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 00:58:15,444 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:15,444 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:58:15,445 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:58:15,454 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/55E6E3B2E8654DEC909CD5ECD02A34DA/AB01FC60F25546BDAC14B7E45ED719D4/.tez/application_1463533095444_0001 doesn't exist and is created
2016-05-18 00:58:15,468 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:58:15,468 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:58:15,469 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/55E6E3B2E8654DEC909CD5ECD02A34DA/AB01FC60F25546BDAC14B7E45ED719D4/.tez/application_1463533095444_0001_wd
2016-05-18 00:58:15,479 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463533095444_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:58:15,479 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:15,479 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:58:15,481 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:58:15,481 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:58:15,481 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:58:15,482 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:58:15,483 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:58:15,485 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:58:15,485 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:58:15,485 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:58:15,485 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:58:15,485 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:58:15,486 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:58:15,486 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:58:15,486 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:58:15,486 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:15,487 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:58:15,487 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:58:15,487 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:58:15,489 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:58:15,490 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:58:15,491 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:58:15,491 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/55E6E3B2E8654DEC909CD5ECD02A34DA/AB01FC60F25546BDAC14B7E45ED719D4/.tez/application_1463533095444_0001_wd/localmode-log-dir
2016-05-18 00:58:15,491 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/55E6E3B2E8654DEC909CD5ECD02A34DA/AB01FC60F25546BDAC14B7E45ED719D4/.tez/application_1463533095444_0001_wd/localmode-log-dir/history.txt.appattempt_1463533095444_0001_000000, maxErrors=10
2016-05-18 00:58:15,492 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/55E6E3B2E8654DEC909CD5ECD02A34DA/AB01FC60F25546BDAC14B7E45ED719D4/.tez/application_1463533095444_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:58:15,493 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463533095444_0001_000000, appSubmitTime=1463533095478, launchTime=1463533095478
2016-05-18 00:58:15,496 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:58:15,499 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:41204
2016-05-18 00:58:15,501 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463533095444
2016-05-18 00:58:15,502 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463533095444_0001_000000, startTime=1463533095502
2016-05-18 00:58:15,502 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:58:15,568 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:58:15,568 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:58:15,569 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1, applicationId=application_1463533095444_0001, dagName=[55E6E3B2E8654DEC909CD5ECD02A34DA/AB01FC60F25546BDAC14B7E45ED719D4] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1
2016-05-18 00:58:15,585 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [55E6E3B2E8654DEC909CD5ECD02A34DA/AB01FC60F25546BDAC14B7E45ED719D4] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1
2016-05-18 00:58:15,586 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:58:15,586 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463533095444_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/55E6E3B2E8654DEC909CD5ECD02A34DA/AB01FC60F25546BDAC14B7E45ED719D4/.tez/application_1463533095444_0001_wd/localmode-log-dir/dag_1463533095444_0001_1.dot
2016-05-18 00:58:15,587 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/55E6E3B2E8654DEC909CD5ECD02A34DA/AB01FC60F25546BDAC14B7E45ED719D4/.tez/application_1463533095444_0001_wd/localmode-log-dir/dag_1463533095444_0001_1-tez-dag.pb.txt
2016-05-18 00:58:15,590 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533095444_0001_1
2016-05-18 00:58:15,590 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:15,590 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [55E6E3B2E8654DEC909CD5ECD02A34DA/AB01FC60F25546BDAC14B7E45ED719D4] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1
2016-05-18 00:58:15 Running Dag: dag_1463533095444_0001_1
2016-05-18 00:58:15,598 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533095444_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463533095444_0001_1, submitTime=1463533095586
2016-05-18 00:58:15,599 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/55E6E3B2E8654DEC909CD5ECD02A34DA/AB01FC60F25546BDAC14B7E45ED719D4/.tez/application_1463533095444_0001/tez-conf.pb" } size: 36913 timestamp: 1463533095000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:58:15,599 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8]
2016-05-18 00:58:15,600 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8]
2016-05-18 00:58:15,600 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:58:15,600 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:58:15,600 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533095444_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463533095444_0001_1, initTime=1463533095600
2016-05-18 00:58:15,601 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533095444_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:58:15,601 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:58:15,601 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533095444_0001_1][Event:DAG_STARTED]: dagID=dag_1463533095444_0001_1, startTime=1463533095601
2016-05-18 00:58:15,601 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533095444_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:58:15,602 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: B6A1D49C64DD49A08D98ED09A008E5F8 : {1849CA2AF83D4807AC125B9FA01DB6A4={InputName=1849CA2AF83D4807AC125B9FA01DB6A4}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:58:15,602 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 1849CA2AF83D4807AC125B9FA01DB6A4, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:58:15,602 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8]
2016-05-18 00:58:15,602 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8]
2016-05-18 00:58:15,602 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8]
2016-05-18 00:58:15,602 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8]
2016-05-18 00:58:15,601 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1, applicationId=application_1463533095444_0001, dagName=[55E6E3B2E8654DEC909CD5ECD02A34DA/AB01FC60F25546BDAC14B7E45ED719D4] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1
2016-05-18 00:58:15,603 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 1849CA2AF83D4807AC125B9FA01DB6A4 on vertex vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8]
2016-05-18 00:58:15,603 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:58:15,604 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 1849CA2AF83D4807AC125B9FA01DB6A4 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:58:15,605 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:58:15,605 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 00:58:15,605 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:58:15,605 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:58:15,606 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 1849CA2AF83D4807AC125B9FA01DB6A4 on vertex vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8]
2016-05-18 00:58:15,608 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:15,608 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {1849CA2AF83D4807AC125B9FA01DB6A4=forAllWorkUnits=true, update=[1]}
2016-05-18 00:58:15,609 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8] parallelism set to 1
2016-05-18 00:58:15,609 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:58:15,609 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8], numAdditionalOutputs=1
2016-05-18 00:58:15,609 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=FD9DEC2BBEC949779346F0D7844DDC71, vertex=vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:58:15,611 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for B6A1D49C64DD49A08D98ED09A008E5F8:FD9DEC2BBEC949779346F0D7844DDC71 using oldmapred API
2016-05-18 00:58:15,611 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=FD9DEC2BBEC949779346F0D7844DDC71, vertexName=B6A1D49C64DD49A08D98ED09A008E5F8, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:58:15,612 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533095444_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B6A1D49C64DD49A08D98ED09A008E5F8, vertexId=vertex_1463533095444_0001_1_00, initRequestedTime=1463533095602, initedTime=1463533095609, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:58:15,612 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8] with distanceFromRoot: 0
2016-05-18 00:58:15,612 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:58:15,613 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533095444_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463533095444_0001_1_00, startRequestedTime=1463533095603, startedTime=1463533095613
2016-05-18 00:58:15,613 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533095444_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463533095444_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:58:15,614 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:58:15,614 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in B6A1D49C64DD49A08D98ED09A008E5F8
2016-05-18 00:58:15,614 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8]
2016-05-18 00:58:15,615 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533095444_0001_1][Event:TASK_STARTED]: vertexName=B6A1D49C64DD49A08D98ED09A008E5F8, taskId=task_1463533095444_0001_1_00_000000, scheduledTime=1463533095614, launchTime=1463533095614
2016-05-18 00:58:15,616 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8]
2016-05-18 00:58:15,617 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:58:15,619 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463533095444_0001_00_000001, launchTime=1463533095619
2016-05-18 00:58:15,619 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533095444_0001_00_000001
2016-05-18 00:58:15,620 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463533095444_0001, with dag id: dag_1463533095444_0001_1
2016-05-18 00:58:15,622 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533095444_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463533095444_0001_1_00_000000_0
2016-05-18 00:58:15,622 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463533095444_0001_1_00_000000_0] started. Is using containerId: [container_1463533095444_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:58:15,622 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533095444_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B6A1D49C64DD49A08D98ED09A008E5F8, taskAttemptId=attempt_1463533095444_0001_1_00_000000_0, startTime=1463533095620, containerId=container_1463533095444_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:58:15 Starting to run new task attempt: attempt_1463533095444_0001_1_00_000000_0
2016-05-18 00:58:15,622 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463533095444_0001_1_00_000000_0
2016-05-18 00:58:15,622 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:15,623 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:58:15,624 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [55E6E3B2E8654DEC909CD5ECD02A34DA/AB01FC60F25546BDAC14B7E45ED719D4] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1, VertexName: B6A1D49C64DD49A08D98ED09A008E5F8, VertexParallelism: 1, TaskAttemptID:attempt_1463533095444_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=1849CA2AF83D4807AC125B9FA01DB6A4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FD9DEC2BBEC949779346F0D7844DDC71, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:58:15,624 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:58:15,625 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463533095444_0001_1_00_000000_0
2016-05-18 00:58:15,626 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463533095444_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:58:15,626 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463533095444_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:58:15,628 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:58:15,630 INFO input.MRInput (MRInput.java:initialize(440)) - 1849CA2AF83D4807AC125B9FA01DB6A4 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:58:15,631 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 1849CA2AF83D4807AC125B9FA01DB6A4
2016-05-18 00:58:15,635 INFO output.MROutput (MROutput.java:initialize(421)) - FD9DEC2BBEC949779346F0D7844DDC71: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:58:15,635 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B6A1D49C64DD49A08D98ED09A008E5F8, ordinal: 0
2016-05-18 00:58:15,636 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B6A1D49C64DD49A08D98ED09A008E5F8, mem on start (mb), free: 256, total: 383, max: 672
2016-05-18 00:58:15,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:58:15,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:58:15,637 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:58:15,637 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:58:15,638 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[FD9DEC2BBEC949779346F0D7844DDC71:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [1849CA2AF83D4807AC125B9FA01DB6A4:INPUT:0:org.apache.tez.mapreduce.input.MRInput]
2016-05-18 00:58:15,638 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:58:15,639 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:58:15,639 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:58:15,639 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[FD9DEC2BBEC949779346F0D7844DDC71:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [1849CA2AF83D4807AC125B9FA01DB6A4:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0]
2016-05-18 00:58:15,640 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:58:15,640 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 1849CA2AF83D4807AC125B9FA01DB6A4
2016-05-18 00:58:15,640 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 1849CA2AF83D4807AC125B9FA01DB6A4 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:58:15,641 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:58:15,642 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:58:15,642 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:58:15,642 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463533095444_0001_1_00_000000_0
2016-05-18 00:58:15,644 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 00:58:15,644 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 1849CA2AF83D4807AC125B9FA01DB6A4 initialized RecordReader from event
2016-05-18 00:58:15,645 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:15,645 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 1849CA2AF83D4807AC125B9FA01DB6A4
2016-05-18 00:58:15,646 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_UNKNOWN_-1"], id: FD9DEC2BBEC949779346F0D7844DDC71
2016-05-18 00:58:15,646 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_UNKNOWN_-1"]
2016-05-18 00:58:15,646 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 00:58:15,647 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B6A1D49C64DD49A08D98ED09A008E5F8, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:58:15,649 INFO output.MROutput (MROutput.java:close(526)) - FD9DEC2BBEC949779346F0D7844DDC71 closed
2016-05-18 00:58:15,649 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463533095444_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:58:15,650 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463533095444_0001_1_00_000000_0
2016-05-18 00:58:15,650 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B6A1D49C64DD49A08D98ED09A008E5F8, mem on close (mb), free: 255, total: 383, max: 672
2016-05-18 00:58:15,651 INFO output.MROutput (MROutput.java:close(526)) - FD9DEC2BBEC949779346F0D7844DDC71 closed
2016-05-18 00:58:15,651 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:58:15 Completed running task attempt: attempt_1463533095444_0001_1_00_000000_0
2016-05-18 00:58:15,652 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463533095444_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:58:15,652 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463533095444_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:58:15,654 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463533095444_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463533095647, Process_Duration=3, Process_End_Time=1463533095650, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:58:15,655 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463533095444_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:15,655 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:58:15,655 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533095444_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B6A1D49C64DD49A08D98ED09A008E5F8, taskAttemptId=attempt_1463533095444_0001_1_00_000000_0, creationTime=1463533095615, allocationTime=1463533095618, startTime=1463533095620, finishTime=1463533095655, timeTaken=35, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:58:15,655 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:58:15,655 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:15,656 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463533095444_0001_1_00_000000_0
2016-05-18 00:58:15,656 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463533095444_0001_00_000001
2016-05-18 00:58:15,656 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533095444_0001_1][Event:TASK_FINISHED]: vertexName=B6A1D49C64DD49A08D98ED09A008E5F8, taskId=task_1463533095444_0001_1_00_000000, startTime=1463533095620, finishTime=1463533095656, timeTaken=36, status=SUCCEEDED, successfulAttemptID=attempt_1463533095444_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533095647, Process_Duration=3, Process_End_Time=1463533095650, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:15,657 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:58:15,657 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8]
2016-05-18 00:58:15,658 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533095444_0001_1][Event:VERTEX_FINISHED]: vertexName=B6A1D49C64DD49A08D98ED09A008E5F8, vertexId=vertex_1463533095444_0001_1_00, initRequestedTime=1463533095602, initedTime=1463533095609, startRequestedTime=1463533095603, startedTime=1463533095613, finishTime=1463533095657, timeTaken=44, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533095647, Process_Duration=3, Process_End_Time=1463533095650, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463533095620, firstTasksToStart=[ task_1463533095444_0001_1_00_000000 ], lastTaskFinishTime=1463533095655, lastTasksToFinish=[ task_1463533095444_0001_1_00_000000 ], minTaskDuration=35, maxTaskDuration=35, avgTaskDuration=35.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463533095444_0001_1_00_000000 ], longestDurationTasks=[ task_1463533095444_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:58:15,658 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:58:15,659 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533095444_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463533095444_0001_00_000001, stoppedTime=1463533095659, exitStatus=0
2016-05-18 00:58:15,659 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:58:15,659 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:15,659 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463533095444_0001_1
2016-05-18 00:58:15,659 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533095444_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463533095444_0001_1
2016-05-18 00:58:15,659 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: FD9DEC2BBEC949779346F0D7844DDC71 for vertex: vertex_1463533095444_0001_1_00 [B6A1D49C64DD49A08D98ED09A008E5F8], outputName: FD9DEC2BBEC949779346F0D7844DDC71
2016-05-18 00:58:15,659 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533095444_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:58:15,660 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463533095444_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:58:15,666 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:FD9DEC2BBEC949779346F0D7844DDC71 of vertex/vertexGroup:B6A1D49C64DD49A08D98ED09A008E5F8 isVertexGroupOutput:false
2016-05-18 00:58:15,666 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:58:15,666 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463533095444_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:58:15,666 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463533095444_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:58:15,667 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463533095444_0001_00_000001, Exiting
2016-05-18 00:58:15,667 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463533095444_0001_00_000001
2016-05-18 00:58:15,667 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463533095444_0001_00_000001
2016-05-18 00:58:15,667 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463533095444_0001_1, queueSize=0
2016-05-18 00:58:15,668 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463533095444_0001_1][Event:DAG_FINISHED]: dagId=dag_1463533095444_0001_1, startTime=1463533095601, finishTime=1463533095666, timeTaken=65, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=120, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463533095647, Process_Duration=3, Process_End_Time=1463533095650, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:58:15,668 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463533095444_0001_1 finished with state: SUCCEEDED
2016-05-18 00:58:15,668 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463533095444_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:58:15 Completed Dag: dag_1463533095444_0001_1
2016-05-18 00:58:15,668 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463533095444_0001_1, dagState=SUCCEEDED
2016-05-18 00:58:15,668 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463533095444_0001_1_post
2016-05-18 00:58:15,668 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:58:15,669 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:58:15,669 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:58:15,669 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[55E6E3B2E8654DEC909CD5ECD02A34DA/AB01FC60F25546BDAC14B7E45ED719D4] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1, with id=dag_1463533095444_0001_1
2016-05-18 00:58:15,669 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[55E6E3B2E8654DEC909CD5ECD02A34DA/AB01FC60F25546BDAC14B7E45ED719D4] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1, with id=dag_1463533095444_0001_1
2016-05-18 00:58:15,669 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463533095444_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:58:15,670 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463533095444_0001_00_000001 completed successfully
2016-05-18 00:58:15,680 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1, applicationId=application_1463533095444_0001
2016-05-18 00:58:15,681 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:58:15,681 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:58:15,681 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:58:15,681 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:58:15,681 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:58:15,682 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:58:15,682 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:58:15,682 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:58:15,682 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:58:15,682 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:58:15,683 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:58:15,683 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:58:15,683 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:58:15,684 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:58:15,687 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/55E6E3B2E8654DEC909CD5ECD02A34DA/AB01FC60F25546BDAC14B7E45ED719D4/.tez/application_1463533095444_0001
2016-05-18 00:58:15,688 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_UNKNOWN_-1/_temporary
2016-05-18 00:58:15,688 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.265
2016-05-18 00:58:15,761 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_UNKNOWN_-1
2016-05-18 00:58:15,788 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:58:15,788 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:58:15,788 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:58:15,789 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:58:15,818 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-05-18 00:58:15,818 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-05-18 00:58:15,819 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 00:58:15,819 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 00:58:15,819 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_UNKNOWN_'field2'