2016-05-18 02:07:10,184 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(197)) - found 1 test platforms from classpath
2016-05-18 02:07:10,208 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(246)) - adding test: cascading.detail.EveryEveryPipeAssemblyPlatformTest, with platform: hadoop2-tez
2016-05-18 02:07:10,659 INFO tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-05-18 02:07:11,159 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-05-18 02:07:11,452 INFO platform.TestPlatform (TestPlatform.java:getGlobalProperties(75)) - platform property overrides:
2016-05-18 02:07:11,662 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:11,664 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:11,748 INFO property.AppProps (AppProps.java:getAppID(177)) - using app.id: 40D0995C9D584C5A9D340EA7D51CEFFF
2016-05-18 02:07:11,980 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.071
2016-05-18 02:07:11,993 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.085
2016-05-18 02:07:11,994 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:11,994 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:11,995 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:12,127 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:12,130 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_-1
2016-05-18 02:07:12,320 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.1.0
2016-05-18 02:07:12,323 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:12,324 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:12,324 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_-1"]
2016-05-18 02:07:12,324 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:12,325 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:12,325 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:12,326 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...d'_RESULTS.-1_'field2'_-1
2016-05-18 02:07:12,326 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:12,347 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:12,521 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:12,528 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:12,528 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:12,577 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/49E612E3C9164DD983D2DC9FF816FFCE/D33596E0FECF4AF6BE334348203531EF/.tez/application_1463537232527_0001 doesn't exist and is created
2016-05-18 02:07:12,685 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:12,686 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:12,687 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/49E612E3C9164DD983D2DC9FF816FFCE/D33596E0FECF4AF6BE334348203531EF/.tez/application_1463537232527_0001_wd
2016-05-18 02:07:12,753 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537232527_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:12,769 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:12,770 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:12,783 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:12,783 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:12,784 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:12,784 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:12,787 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-05-18 02:07:12,787 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-05-18 02:07:12,787 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:12,809 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:12,835 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:12,837 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:12,839 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:12,840 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:12,841 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:12,841 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:12,842 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:12,844 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:12,845 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:12,845 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:12,854 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:12,858 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:12,879 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:12,881 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:12,882 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:12,886 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/49E612E3C9164DD983D2DC9FF816FFCE/D33596E0FECF4AF6BE334348203531EF/.tez/application_1463537232527_0001_wd/localmode-log-dir
2016-05-18 02:07:12,886 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/49E612E3C9164DD983D2DC9FF816FFCE/D33596E0FECF4AF6BE334348203531EF/.tez/application_1463537232527_0001_wd/localmode-log-dir/history.txt.appattempt_1463537232527_0001_000000, maxErrors=10
2016-05-18 02:07:12,887 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/49E612E3C9164DD983D2DC9FF816FFCE/D33596E0FECF4AF6BE334348203531EF/.tez/application_1463537232527_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:12,887 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-05-18 02:07:12,888 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-05-18 02:07:12,888 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:12,890 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537232527_0001_000000, appSubmitTime=1463537232731, launchTime=1463537232751
2016-05-18 02:07:12,899 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:12,963 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:56887
2016-05-18 02:07:12,964 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537232527
2016-05-18 02:07:12,983 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537232527_0001_000000, startTime=1463537232982
2016-05-18 02:07:12,983 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:12,988 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:13,013 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:13,013 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...d'_RESULTS.-1_'field2'_-1, applicationId=application_1463537232527_0001, dagName=[49E612E3C9164DD983D2DC9FF816FFCE/D33596E0FECF4AF6BE334348203531EF] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1
2016-05-18 02:07:13,225 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [49E612E3C9164DD983D2DC9FF816FFCE/D33596E0FECF4AF6BE334348203531EF] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1
2016-05-18 02:07:13,235 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:13,294 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537232527_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/49E612E3C9164DD983D2DC9FF816FFCE/D33596E0FECF4AF6BE334348203531EF/.tez/application_1463537232527_0001_wd/localmode-log-dir/dag_1463537232527_0001_1.dot
2016-05-18 02:07:13,297 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/49E612E3C9164DD983D2DC9FF816FFCE/D33596E0FECF4AF6BE334348203531EF/.tez/application_1463537232527_0001_wd/localmode-log-dir/dag_1463537232527_0001_1-tez-dag.pb.txt
2016-05-18 02:07:13,340 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537232527_0001_1
2016-05-18 02:07:13,340 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:13,340 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [49E612E3C9164DD983D2DC9FF816FFCE/D33596E0FECF4AF6BE334348203531EF] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1
2016-05-18 02:07:13 Running Dag: dag_1463537232527_0001_1
2016-05-18 02:07:13,369 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537232527_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537232527_0001_1, submitTime=1463537233225
2016-05-18 02:07:13,425 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/49E612E3C9164DD983D2DC9FF816FFCE/D33596E0FECF4AF6BE334348203531EF/.tez/application_1463537232527_0001/tez-conf.pb" } size: 40947 timestamp: 1463537232000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:13,426 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D]
2016-05-18 02:07:13,426 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D]
2016-05-18 02:07:13,428 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:13,428 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:13,432 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537232527_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537232527_0001_1, initTime=1463537233431
2016-05-18 02:07:13,432 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537232527_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:13,435 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:13,438 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537232527_0001_1][Event:DAG_STARTED]: dagID=dag_1463537232527_0001_1, startTime=1463537233437
2016-05-18 02:07:13,438 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537232527_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:13,438 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: CA358517630741BCB3875ADE93E3D63D : {457B08215BBD49A8937D9E53DBD9FFC1={InputName=457B08215BBD49A8937D9E53DBD9FFC1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:13,439 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 457B08215BBD49A8937D9E53DBD9FFC1, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:13,439 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D]
2016-05-18 02:07:13,448 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...d'_RESULTS.-1_'field2'_-1, applicationId=application_1463537232527_0001, dagName=[49E612E3C9164DD983D2DC9FF816FFCE/D33596E0FECF4AF6BE334348203531EF] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1
2016-05-18 02:07:13,452 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D]
2016-05-18 02:07:13,452 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D]
2016-05-18 02:07:13,455 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D]
2016-05-18 02:07:13,463 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 457B08215BBD49A8937D9E53DBD9FFC1 on vertex vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D]
2016-05-18 02:07:13,478 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:13,484 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 457B08215BBD49A8937D9E53DBD9FFC1 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:13,498 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537232527_0001, with dag id: dag_1463537232527_0001_1
2016-05-18 02:07:13,524 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:13,525 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:13,526 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:13,527 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:13,539 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 457B08215BBD49A8937D9E53DBD9FFC1 on vertex vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D]
2016-05-18 02:07:13,545 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:13,546 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {457B08215BBD49A8937D9E53DBD9FFC1=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:13,557 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D] parallelism set to 1
2016-05-18 02:07:13,557 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:13,564 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D], numAdditionalOutputs=1
2016-05-18 02:07:13,564 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=A8F7A2C0A8354FE0AD0F418CDFB175B1, vertex=vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:13,577 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for CA358517630741BCB3875ADE93E3D63D:A8F7A2C0A8354FE0AD0F418CDFB175B1 using oldmapred API
2016-05-18 02:07:13,578 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=A8F7A2C0A8354FE0AD0F418CDFB175B1, vertexName=CA358517630741BCB3875ADE93E3D63D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:13,589 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537232527_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CA358517630741BCB3875ADE93E3D63D, vertexId=vertex_1463537232527_0001_1_00, initRequestedTime=1463537233438, initedTime=1463537233564, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:13,589 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D] with distanceFromRoot: 0
2016-05-18 02:07:13,589 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:13,591 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537232527_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537232527_0001_1_00, startRequestedTime=1463537233479, startedTime=1463537233589
2016-05-18 02:07:13,593 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537232527_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537232527_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:13,593 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:13,595 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in CA358517630741BCB3875ADE93E3D63D
2016-05-18 02:07:13,595 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D]
2016-05-18 02:07:13,601 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537232527_0001_1][Event:TASK_STARTED]: vertexName=CA358517630741BCB3875ADE93E3D63D, taskId=task_1463537232527_0001_1_00_000000, scheduledTime=1463537233599, launchTime=1463537233599
2016-05-18 02:07:13,633 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D]
2016-05-18 02:07:13,684 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:13,702 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537232527_0001_00_000001, launchTime=1463537233701
2016-05-18 02:07:13,708 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537232527_0001_00_000001
2016-05-18 02:07:13,711 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537232527_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537232527_0001_1_00_000000_0
2016-05-18 02:07:13,713 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537232527_0001_1_00_000000_0] started. Is using containerId: [container_1463537232527_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:13 Starting to run new task attempt: attempt_1463537232527_0001_1_00_000000_0
2016-05-18 02:07:13,714 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537232527_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CA358517630741BCB3875ADE93E3D63D, taskAttemptId=attempt_1463537232527_0001_1_00_000000_0, startTime=1463537233711, containerId=container_1463537232527_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:13,714 INFO counters.Limits (Limits.java:ensureInitialized(60)) - Counter limits initialized with parameters: GROUP_NAME_MAX=256, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-05-18 02:07:13,714 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537232527_0001_1_00_000000_0
2016-05-18 02:07:13,715 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:13,717 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:13,730 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [49E612E3C9164DD983D2DC9FF816FFCE/D33596E0FECF4AF6BE334348203531EF] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1, VertexName: CA358517630741BCB3875ADE93E3D63D, VertexParallelism: 1, TaskAttemptID:attempt_1463537232527_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=457B08215BBD49A8937D9E53DBD9FFC1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A8F7A2C0A8354FE0AD0F418CDFB175B1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:13,732 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:13,737 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537232527_0001_1_00_000000_0
2016-05-18 02:07:13,741 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537232527_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:13,743 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537232527_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:13,774 INFO input.MRInput (MRInput.java:initialize(440)) - 457B08215BBD49A8937D9E53DBD9FFC1 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:13,775 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 457B08215BBD49A8937D9E53DBD9FFC1
2016-05-18 02:07:13,777 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:13,793 INFO output.MROutput (MROutput.java:initialize(421)) - A8F7A2C0A8354FE0AD0F418CDFB175B1: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:13,795 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CA358517630741BCB3875ADE93E3D63D, ordinal: 0
2016-05-18 02:07:13,796 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CA358517630741BCB3875ADE93E3D63D, mem on start (mb), free: 178, total: 281, max: 672
2016-05-18 02:07:13,796 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:13,797 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:13,797 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:13,797 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:13,798 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[457B08215BBD49A8937D9E53DBD9FFC1:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [A8F7A2C0A8354FE0AD0F418CDFB175B1:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:13,803 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:13,807 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:13,807 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:13,808 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[457B08215BBD49A8937D9E53DBD9FFC1:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [A8F7A2C0A8354FE0AD0F418CDFB175B1:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:13,809 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:13,810 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 457B08215BBD49A8937D9E53DBD9FFC1 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:13,810 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 457B08215BBD49A8937D9E53DBD9FFC1
2016-05-18 02:07:13,810 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:13,812 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:13,812 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:13,815 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537232527_0001_1_00_000000_0
2016-05-18 02:07:13,837 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:13,838 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 457B08215BBD49A8937D9E53DBD9FFC1 initialized RecordReader from event
2016-05-18 02:07:13,866 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:13,892 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 457B08215BBD49A8937D9E53DBD9FFC1
2016-05-18 02:07:13,893 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_-1"], id: A8F7A2C0A8354FE0AD0F418CDFB175B1
2016-05-18 02:07:13,893 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_-1"]
2016-05-18 02:07:13,897 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:13,899 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CA358517630741BCB3875ADE93E3D63D, all 1 inputs ready in: 00:00:00.001
2016-05-18 02:07:13,908 INFO output.MROutput (MROutput.java:close(526)) - A8F7A2C0A8354FE0AD0F418CDFB175B1 closed
2016-05-18 02:07:13,910 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537232527_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:13,912 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537232527_0001_1_00_000000_0
2016-05-18 02:07:13,912 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CA358517630741BCB3875ADE93E3D63D, mem on close (mb), free: 171, total: 281, max: 672
2016-05-18 02:07:13,913 INFO output.MROutput (MROutput.java:close(526)) - A8F7A2C0A8354FE0AD0F418CDFB175B1 closed
2016-05-18 02:07:13,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:13 Completed running task attempt: attempt_1463537232527_0001_1_00_000000_0
2016-05-18 02:07:13,915 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537232527_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:13,915 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537232527_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:13,918 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537232527_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537233900, Process_Duration=12, Process_End_Time=1463537233912, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:13,924 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537232527_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:13,924 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:13,925 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537232527_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CA358517630741BCB3875ADE93E3D63D, taskAttemptId=attempt_1463537232527_0001_1_00_000000_0, creationTime=1463537233624, allocationTime=1463537233692, startTime=1463537233711, finishTime=1463537233923, timeTaken=212, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:13,927 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:13,928 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:13,930 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537232527_0001_1_00_000000_0
2016-05-18 02:07:13,931 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537232527_0001_00_000001
2016-05-18 02:07:13,937 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537232527_0001_1][Event:TASK_FINISHED]: vertexName=CA358517630741BCB3875ADE93E3D63D, taskId=task_1463537232527_0001_1_00_000000, startTime=1463537233711, finishTime=1463537233933, timeTaken=222, status=SUCCEEDED, successfulAttemptID=attempt_1463537232527_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537233900, Process_Duration=12, Process_End_Time=1463537233912, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:13,944 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:13,944 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D]
2016-05-18 02:07:13,963 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537232527_0001_1][Event:VERTEX_FINISHED]: vertexName=CA358517630741BCB3875ADE93E3D63D, vertexId=vertex_1463537232527_0001_1_00, initRequestedTime=1463537233438, initedTime=1463537233564, startRequestedTime=1463537233479, startedTime=1463537233589, finishTime=1463537233945, timeTaken=356, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537233900, Process_Duration=12, Process_End_Time=1463537233912, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537233711, firstTasksToStart=[ task_1463537232527_0001_1_00_000000 ], lastTaskFinishTime=1463537233923, lastTasksToFinish=[ task_1463537232527_0001_1_00_000000 ], minTaskDuration=212, maxTaskDuration=212, avgTaskDuration=212.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537232527_0001_1_00_000000 ], longestDurationTasks=[ task_1463537232527_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:13,964 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:13,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537232527_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537232527_0001_00_000001, stoppedTime=1463537233966, exitStatus=0
2016-05-18 02:07:13,968 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:13,968 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:13,969 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537232527_0001_1
2016-05-18 02:07:13,972 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537232527_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537232527_0001_1
2016-05-18 02:07:13,973 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: A8F7A2C0A8354FE0AD0F418CDFB175B1 for vertex: vertex_1463537232527_0001_1_00 [CA358517630741BCB3875ADE93E3D63D], outputName: A8F7A2C0A8354FE0AD0F418CDFB175B1
2016-05-18 02:07:13,976 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537232527_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:13,977 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537232527_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:13,990 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:A8F7A2C0A8354FE0AD0F418CDFB175B1 of vertex/vertexGroup:CA358517630741BCB3875ADE93E3D63D isVertexGroupOutput:false
2016-05-18 02:07:13,990 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:13,995 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537232527_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:13,995 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537232527_0001_00_000001: 61 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:13,995 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537232527_0001_00_000001, Exiting
2016-05-18 02:07:13,995 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537232527_0001_00_000001
2016-05-18 02:07:13,996 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537232527_0001_00_000001
2016-05-18 02:07:13,998 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537232527_0001_1, queueSize=0
2016-05-18 02:07:13,999 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537232527_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537232527_0001_1, startTime=1463537233435, finishTime=1463537233990, timeTaken=555, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=1310, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537233900, Process_Duration=12, Process_End_Time=1463537233912, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:14,000 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537232527_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:14,001 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537232527_0001_00_000001 completed successfully
2016-05-18 02:07:14,001 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537232527_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:14,003 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537232527_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:14 Completed Dag: dag_1463537232527_0001_1
2016-05-18 02:07:14,004 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537232527_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:14,005 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537232527_0001_1_post
2016-05-18 02:07:14,005 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:14,005 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:14,006 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:14,007 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[49E612E3C9164DD983D2DC9FF816FFCE/D33596E0FECF4AF6BE334348203531EF] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1, with id=dag_1463537232527_0001_1
2016-05-18 02:07:14,013 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[49E612E3C9164DD983D2DC9FF816FFCE/D33596E0FECF4AF6BE334348203531EF] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1, with id=dag_1463537232527_0001_1
2016-05-18 02:07:14,022 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...d'_RESULTS.-1_'field2'_-1, applicationId=application_1463537232527_0001
2016-05-18 02:07:14,027 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:14,027 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:14,027 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:14,028 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:14,028 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:14,031 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:14,033 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:14,033 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:14,033 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:14,033 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:14,034 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:14,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:14,035 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:14,035 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:14,040 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/49E612E3C9164DD983D2DC9FF816FFCE/D33596E0FECF4AF6BE334348203531EF/.tez/application_1463537232527_0001
2016-05-18 02:07:14,042 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_-1/_temporary
2016-05-18 02:07:14,042 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:01.717
2016-05-18 02:07:14,217 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_'field2'_-1
2016-05-18 02:07:14,253 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:14,254 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:14,323 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-05-18 02:07:14,338 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.046
2016-05-18 02:07:14,339 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:14,340 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:14,340 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:14,362 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:14,363 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field2'
2016-05-18 02:07:14,396 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:14,397 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:14,397 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field2'"]
2016-05-18 02:07:14,398 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:14,398 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:14,398 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:14,399 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ULTS.-1_'field2'_'field2'
2016-05-18 02:07:14,400 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:14,430 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:14,437 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:14,438 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:14,439 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:14,453 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9378848FF522467EAE052A9517C8D32E/F9A3FE91DD8644A0A7E7CBFABAB6DF99/.tez/application_1463537234438_0001 doesn't exist and is created
2016-05-18 02:07:14,484 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:14,484 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:14,485 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9378848FF522467EAE052A9517C8D32E/F9A3FE91DD8644A0A7E7CBFABAB6DF99/.tez/application_1463537234438_0001_wd
2016-05-18 02:07:14,504 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537234438_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:14,504 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:14,504 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:14,509 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:14,509 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:14,509 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:14,509 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:14,510 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:14,514 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:14,514 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:14,514 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:14,515 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:14,516 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:14,516 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:14,517 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:14,517 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:14,517 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:14,517 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:14,518 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:14,518 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:14,524 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:14,525 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:14,526 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:14,526 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9378848FF522467EAE052A9517C8D32E/F9A3FE91DD8644A0A7E7CBFABAB6DF99/.tez/application_1463537234438_0001_wd/localmode-log-dir
2016-05-18 02:07:14,527 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9378848FF522467EAE052A9517C8D32E/F9A3FE91DD8644A0A7E7CBFABAB6DF99/.tez/application_1463537234438_0001_wd/localmode-log-dir/history.txt.appattempt_1463537234438_0001_000000, maxErrors=10
2016-05-18 02:07:14,527 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9378848FF522467EAE052A9517C8D32E/F9A3FE91DD8644A0A7E7CBFABAB6DF99/.tez/application_1463537234438_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:14,528 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537234438_0001_000000, appSubmitTime=1463537234502, launchTime=1463537234502
2016-05-18 02:07:14,534 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:14,541 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:54684
2016-05-18 02:07:14,542 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537234438
2016-05-18 02:07:14,546 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537234438_0001_000000, startTime=1463537234546
2016-05-18 02:07:14,546 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:14,584 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:14,584 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:14,585 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ULTS.-1_'field2'_'field2', applicationId=application_1463537234438_0001, dagName=[9378848FF522467EAE052A9517C8D32E/F9A3FE91DD8644A0A7E7CBFABAB6DF99] every.every/(1/1) ...ULTS.-1_'field2'_'field2'
2016-05-18 02:07:14,608 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [9378848FF522467EAE052A9517C8D32E/F9A3FE91DD8644A0A7E7CBFABAB6DF99] every.every/(1/1) ...ULTS.-1_'field2'_'field2'
2016-05-18 02:07:14,608 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:14,609 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537234438_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9378848FF522467EAE052A9517C8D32E/F9A3FE91DD8644A0A7E7CBFABAB6DF99/.tez/application_1463537234438_0001_wd/localmode-log-dir/dag_1463537234438_0001_1.dot
2016-05-18 02:07:14,610 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9378848FF522467EAE052A9517C8D32E/F9A3FE91DD8644A0A7E7CBFABAB6DF99/.tez/application_1463537234438_0001_wd/localmode-log-dir/dag_1463537234438_0001_1-tez-dag.pb.txt
2016-05-18 02:07:14,615 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537234438_0001_1
2016-05-18 02:07:14,615 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:14,615 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [9378848FF522467EAE052A9517C8D32E/F9A3FE91DD8644A0A7E7CBFABAB6DF99] every.every/(1/1) ...ULTS.-1_'field2'_'field2'
2016-05-18 02:07:14 Running Dag: dag_1463537234438_0001_1
2016-05-18 02:07:14,632 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537234438_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537234438_0001_1, submitTime=1463537234608
2016-05-18 02:07:14,634 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9378848FF522467EAE052A9517C8D32E/F9A3FE91DD8644A0A7E7CBFABAB6DF99/.tez/application_1463537234438_0001/tez-conf.pb" } size: 40947 timestamp: 1463537234000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:14,634 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78]
2016-05-18 02:07:14,635 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78]
2016-05-18 02:07:14,635 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:14,635 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:14,636 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537234438_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537234438_0001_1, initTime=1463537234636
2016-05-18 02:07:14,637 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537234438_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:14,637 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:14,638 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537234438_0001_1][Event:DAG_STARTED]: dagID=dag_1463537234438_0001_1, startTime=1463537234638
2016-05-18 02:07:14,638 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537234438_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:14,638 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: E3265AD13BBF4E5884172F03C855BD78 : {B761F226421F4616A4BC5F2B84D0968A={InputName=B761F226421F4616A4BC5F2B84D0968A}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:14,638 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: B761F226421F4616A4BC5F2B84D0968A, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:14,638 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78]
2016-05-18 02:07:14,639 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78]
2016-05-18 02:07:14,639 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78]
2016-05-18 02:07:14,639 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78]
2016-05-18 02:07:14,640 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:14,640 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: B761F226421F4616A4BC5F2B84D0968A on vertex vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78]
2016-05-18 02:07:14,642 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ULTS.-1_'field2'_'field2', applicationId=application_1463537234438_0001, dagName=[9378848FF522467EAE052A9517C8D32E/F9A3FE91DD8644A0A7E7CBFABAB6DF99] every.every/(1/1) ...ULTS.-1_'field2'_'field2'
2016-05-18 02:07:14,650 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input B761F226421F4616A4BC5F2B84D0968A asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:14,656 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:14,657 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:14,657 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:14,658 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:14,659 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: B761F226421F4616A4BC5F2B84D0968A on vertex vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78]
2016-05-18 02:07:14,664 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:14,665 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {B761F226421F4616A4BC5F2B84D0968A=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:14,666 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78] parallelism set to 1
2016-05-18 02:07:14,666 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:14,666 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78], numAdditionalOutputs=1
2016-05-18 02:07:14,666 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=C5AA95B084994B3C861EE8EFCB1D4712, vertex=vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:14,672 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for E3265AD13BBF4E5884172F03C855BD78:C5AA95B084994B3C861EE8EFCB1D4712 using oldmapred API
2016-05-18 02:07:14,673 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=C5AA95B084994B3C861EE8EFCB1D4712, vertexName=E3265AD13BBF4E5884172F03C855BD78, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:14,676 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537234438_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E3265AD13BBF4E5884172F03C855BD78, vertexId=vertex_1463537234438_0001_1_00, initRequestedTime=1463537234638, initedTime=1463537234666, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:14,677 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78] with distanceFromRoot: 0
2016-05-18 02:07:14,677 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:14,678 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537234438_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537234438_0001_1_00, startRequestedTime=1463537234641, startedTime=1463537234677
2016-05-18 02:07:14,678 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537234438_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537234438_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:14,679 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:14,679 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in E3265AD13BBF4E5884172F03C855BD78
2016-05-18 02:07:14,680 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78]
2016-05-18 02:07:14,681 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537234438_0001_1][Event:TASK_STARTED]: vertexName=E3265AD13BBF4E5884172F03C855BD78, taskId=task_1463537234438_0001_1_00_000000, scheduledTime=1463537234680, launchTime=1463537234680
2016-05-18 02:07:14,683 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78]
2016-05-18 02:07:14,685 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:14,692 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537234438_0001_00_000001, launchTime=1463537234691
2016-05-18 02:07:14,692 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537234438_0001_00_000001
2016-05-18 02:07:14,694 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537234438_0001_1_00_000000_0] started. Is using containerId: [container_1463537234438_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:14,695 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537234438_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E3265AD13BBF4E5884172F03C855BD78, taskAttemptId=attempt_1463537234438_0001_1_00_000000_0, startTime=1463537234694, containerId=container_1463537234438_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:14,696 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537234438_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537234438_0001_1_00_000000_0
2016-05-18 02:07:14 Starting to run new task attempt: attempt_1463537234438_0001_1_00_000000_0
2016-05-18 02:07:14,696 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537234438_0001_1_00_000000_0
2016-05-18 02:07:14,696 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:14,697 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:14,699 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [9378848FF522467EAE052A9517C8D32E/F9A3FE91DD8644A0A7E7CBFABAB6DF99] every.every/(1/1) ...ULTS.-1_'field2'_'field2', VertexName: E3265AD13BBF4E5884172F03C855BD78, VertexParallelism: 1, TaskAttemptID:attempt_1463537234438_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B761F226421F4616A4BC5F2B84D0968A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C5AA95B084994B3C861EE8EFCB1D4712, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:14,699 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:14,701 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537234438_0001_1_00_000000_0
2016-05-18 02:07:14,704 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537234438_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:14,704 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537234438_0001, with dag id: dag_1463537234438_0001_1
2016-05-18 02:07:14,705 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537234438_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:14,723 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:14,724 INFO input.MRInput (MRInput.java:initialize(440)) - B761F226421F4616A4BC5F2B84D0968A using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:14,726 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: B761F226421F4616A4BC5F2B84D0968A
2016-05-18 02:07:14,732 INFO output.MROutput (MROutput.java:initialize(421)) - C5AA95B084994B3C861EE8EFCB1D4712: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:14,738 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E3265AD13BBF4E5884172F03C855BD78, ordinal: 0
2016-05-18 02:07:14,738 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E3265AD13BBF4E5884172F03C855BD78, mem on start (mb), free: 176, total: 281, max: 672
2016-05-18 02:07:14,738 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:14,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:14,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:14,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:14,740 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[C5AA95B084994B3C861EE8EFCB1D4712:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [B761F226421F4616A4BC5F2B84D0968A:INPUT:0:org.apache.tez.mapreduce.input.MRInput]
2016-05-18 02:07:14,740 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:14,741 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:14,741 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:14,742 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[C5AA95B084994B3C861EE8EFCB1D4712:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [B761F226421F4616A4BC5F2B84D0968A:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0]
2016-05-18 02:07:14,742 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:14,742 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: B761F226421F4616A4BC5F2B84D0968A being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:14,743 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:14,743 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:14,742 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: B761F226421F4616A4BC5F2B84D0968A
2016-05-18 02:07:14,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:14,745 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537234438_0001_1_00_000000_0
2016-05-18 02:07:14,748 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:14,749 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - B761F226421F4616A4BC5F2B84D0968A initialized RecordReader from event
2016-05-18 02:07:14,752 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:14,753 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: B761F226421F4616A4BC5F2B84D0968A
2016-05-18 02:07:14,754 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field2'"], id: C5AA95B084994B3C861EE8EFCB1D4712
2016-05-18 02:07:14,754 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field2'"]
2016-05-18 02:07:14,755 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:14,755 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E3265AD13BBF4E5884172F03C855BD78, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:14,759 INFO output.MROutput (MROutput.java:close(526)) - C5AA95B084994B3C861EE8EFCB1D4712 closed
2016-05-18 02:07:14,761 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537234438_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:14,762 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537234438_0001_1_00_000000_0
2016-05-18 02:07:14,763 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E3265AD13BBF4E5884172F03C855BD78, mem on close (mb), free: 173, total: 281, max: 672
2016-05-18 02:07:14,763 INFO output.MROutput (MROutput.java:close(526)) - C5AA95B084994B3C861EE8EFCB1D4712 closed
2016-05-18 02:07:14,764 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:14 Completed running task attempt: attempt_1463537234438_0001_1_00_000000_0
2016-05-18 02:07:14,766 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537234438_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:14,767 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537234438_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:14,767 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537234438_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537234756, Process_Duration=6, Process_End_Time=1463537234762, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:14,769 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537234438_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:14,770 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:14,770 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:14,770 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:14,770 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537234438_0001_00_000001
2016-05-18 02:07:14,771 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537234438_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E3265AD13BBF4E5884172F03C855BD78, taskAttemptId=attempt_1463537234438_0001_1_00_000000_0, creationTime=1463537234681, allocationTime=1463537234691, startTime=1463537234694, finishTime=1463537234770, timeTaken=76, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:14,776 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537234438_0001_1_00_000000_0
2016-05-18 02:07:14,777 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537234438_0001_1][Event:TASK_FINISHED]: vertexName=E3265AD13BBF4E5884172F03C855BD78, taskId=task_1463537234438_0001_1_00_000000, startTime=1463537234694, finishTime=1463537234776, timeTaken=82, status=SUCCEEDED, successfulAttemptID=attempt_1463537234438_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537234756, Process_Duration=6, Process_End_Time=1463537234762, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:14,777 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:14,777 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78]
2016-05-18 02:07:14,781 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537234438_0001_1][Event:VERTEX_FINISHED]: vertexName=E3265AD13BBF4E5884172F03C855BD78, vertexId=vertex_1463537234438_0001_1_00, initRequestedTime=1463537234638, initedTime=1463537234666, startRequestedTime=1463537234641, startedTime=1463537234677, finishTime=1463537234777, timeTaken=100, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537234756, Process_Duration=6, Process_End_Time=1463537234762, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537234694, firstTasksToStart=[ task_1463537234438_0001_1_00_000000 ], lastTaskFinishTime=1463537234770, lastTasksToFinish=[ task_1463537234438_0001_1_00_000000 ], minTaskDuration=76, maxTaskDuration=76, avgTaskDuration=76.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537234438_0001_1_00_000000 ], longestDurationTasks=[ task_1463537234438_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:14,782 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:14,782 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537234438_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537234438_0001_00_000001, stoppedTime=1463537234782, exitStatus=0
2016-05-18 02:07:14,783 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:14,783 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:14,783 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537234438_0001_1
2016-05-18 02:07:14,784 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537234438_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537234438_0001_1
2016-05-18 02:07:14,784 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: C5AA95B084994B3C861EE8EFCB1D4712 for vertex: vertex_1463537234438_0001_1_00 [E3265AD13BBF4E5884172F03C855BD78], outputName: C5AA95B084994B3C861EE8EFCB1D4712
2016-05-18 02:07:14,784 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537234438_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:14,788 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537234438_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:14,791 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537234438_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:14,791 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537234438_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:14,792 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537234438_0001_00_000001, Exiting
2016-05-18 02:07:14,792 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537234438_0001_00_000001
2016-05-18 02:07:14,792 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537234438_0001_00_000001
2016-05-18 02:07:14,794 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537234438_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:14,794 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537234438_0001_00_000001 completed successfully
2016-05-18 02:07:14,798 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:C5AA95B084994B3C861EE8EFCB1D4712 of vertex/vertexGroup:E3265AD13BBF4E5884172F03C855BD78 isVertexGroupOutput:false
2016-05-18 02:07:14,798 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:14,800 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537234438_0001_1, queueSize=0
2016-05-18 02:07:14,802 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537234438_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537234438_0001_1, startTime=1463537234637, finishTime=1463537234798, timeTaken=161, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=410, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537234756, Process_Duration=6, Process_End_Time=1463537234762, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:14,802 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537234438_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:14,802 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537234438_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:14 Completed Dag: dag_1463537234438_0001_1
2016-05-18 02:07:14,803 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537234438_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:14,803 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537234438_0001_1_post
2016-05-18 02:07:14,803 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:14,803 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:14,803 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:14,803 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[9378848FF522467EAE052A9517C8D32E/F9A3FE91DD8644A0A7E7CBFABAB6DF99] every.every/(1/1) ...ULTS.-1_'field2'_'field2', with id=dag_1463537234438_0001_1
2016-05-18 02:07:14,804 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[9378848FF522467EAE052A9517C8D32E/F9A3FE91DD8644A0A7E7CBFABAB6DF99] every.every/(1/1) ...ULTS.-1_'field2'_'field2', with id=dag_1463537234438_0001_1
2016-05-18 02:07:14,810 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ULTS.-1_'field2'_'field2', applicationId=application_1463537234438_0001
2016-05-18 02:07:14,815 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:14,815 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:14,815 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:14,815 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:14,815 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:14,816 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:14,816 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:14,816 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:14,817 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:14,817 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:14,817 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:14,818 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:14,818 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:14,820 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:14,824 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9378848FF522467EAE052A9517C8D32E/F9A3FE91DD8644A0A7E7CBFABAB6DF99/.tez/application_1463537234438_0001
2016-05-18 02:07:14,825 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field2'/_temporary
2016-05-18 02:07:14,826 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.429
2016-05-18 02:07:14,931 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_'field2'_'field2'
2016-05-18 02:07:14,996 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:14,997 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:15,067 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.034
2016-05-18 02:07:15,075 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.042
2016-05-18 02:07:15,076 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:15,077 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:15,077 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:15,100 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:15,101 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field'
2016-05-18 02:07:15,127 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:15,127 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:15,128 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field'"]
2016-05-18 02:07:15,128 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:15,128 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:15,129 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:15,131 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...SULTS.-1_'field2'_'field'
2016-05-18 02:07:15,131 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:15,149 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:15,154 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:15,155 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:15,155 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:15,165 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/41C7E84CBB66407F9E6ACD865387F227/DE736503824045B4BD228C35BA2E7B28/.tez/application_1463537235155_0001 doesn't exist and is created
2016-05-18 02:07:15,196 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:15,197 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:15,198 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/41C7E84CBB66407F9E6ACD865387F227/DE736503824045B4BD228C35BA2E7B28/.tez/application_1463537235155_0001_wd
2016-05-18 02:07:15,214 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537235155_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:15,215 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:15,215 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:15,217 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:15,217 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:15,217 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:15,217 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:15,218 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:15,220 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:15,221 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:15,221 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:15,221 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:15,221 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:15,221 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:15,221 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:15,221 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:15,221 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:15,222 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:15,222 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:15,222 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:15,224 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:15,225 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:15,225 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:15,226 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/41C7E84CBB66407F9E6ACD865387F227/DE736503824045B4BD228C35BA2E7B28/.tez/application_1463537235155_0001_wd/localmode-log-dir
2016-05-18 02:07:15,226 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/41C7E84CBB66407F9E6ACD865387F227/DE736503824045B4BD228C35BA2E7B28/.tez/application_1463537235155_0001_wd/localmode-log-dir/history.txt.appattempt_1463537235155_0001_000000, maxErrors=10
2016-05-18 02:07:15,226 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/41C7E84CBB66407F9E6ACD865387F227/DE736503824045B4BD228C35BA2E7B28/.tez/application_1463537235155_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:15,227 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537235155_0001_000000, appSubmitTime=1463537235214, launchTime=1463537235214
2016-05-18 02:07:15,235 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:15,239 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:34531
2016-05-18 02:07:15,240 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537235155
2016-05-18 02:07:15,242 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537235155_0001_000000, startTime=1463537235242
2016-05-18 02:07:15,242 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:15,297 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:15,297 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:15,297 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...SULTS.-1_'field2'_'field', applicationId=application_1463537235155_0001, dagName=[41C7E84CBB66407F9E6ACD865387F227/DE736503824045B4BD228C35BA2E7B28] every.every/(1/1) ...SULTS.-1_'field2'_'field'
2016-05-18 02:07:15,320 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [41C7E84CBB66407F9E6ACD865387F227/DE736503824045B4BD228C35BA2E7B28] every.every/(1/1) ...SULTS.-1_'field2'_'field'
2016-05-18 02:07:15,321 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:15,322 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537235155_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/41C7E84CBB66407F9E6ACD865387F227/DE736503824045B4BD228C35BA2E7B28/.tez/application_1463537235155_0001_wd/localmode-log-dir/dag_1463537235155_0001_1.dot
2016-05-18 02:07:15,323 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/41C7E84CBB66407F9E6ACD865387F227/DE736503824045B4BD228C35BA2E7B28/.tez/application_1463537235155_0001_wd/localmode-log-dir/dag_1463537235155_0001_1-tez-dag.pb.txt
2016-05-18 02:07:15,327 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537235155_0001_1
2016-05-18 02:07:15,327 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:15,327 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [41C7E84CBB66407F9E6ACD865387F227/DE736503824045B4BD228C35BA2E7B28] every.every/(1/1) ...SULTS.-1_'field2'_'field'
2016-05-18 02:07:15 Running Dag: dag_1463537235155_0001_1
2016-05-18 02:07:15,341 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235155_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537235155_0001_1, submitTime=1463537235320
2016-05-18 02:07:15,342 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/41C7E84CBB66407F9E6ACD865387F227/DE736503824045B4BD228C35BA2E7B28/.tez/application_1463537235155_0001/tez-conf.pb" } size: 40947 timestamp: 1463537235000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:15,342 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246]
2016-05-18 02:07:15,343 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246]
2016-05-18 02:07:15,343 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:15,343 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:15,343 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235155_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537235155_0001_1, initTime=1463537235343
2016-05-18 02:07:15,344 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537235155_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:15,344 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:15,345 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235155_0001_1][Event:DAG_STARTED]: dagID=dag_1463537235155_0001_1, startTime=1463537235345
2016-05-18 02:07:15,345 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537235155_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:15,345 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: A15449096F4640BA93ED31492B13D246 : {5EF3E809A62D46C69A57A2429F58345C={InputName=5EF3E809A62D46C69A57A2429F58345C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:15,345 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 5EF3E809A62D46C69A57A2429F58345C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:15,346 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246]
2016-05-18 02:07:15,346 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246]
2016-05-18 02:07:15,346 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246]
2016-05-18 02:07:15,346 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...SULTS.-1_'field2'_'field', applicationId=application_1463537235155_0001, dagName=[41C7E84CBB66407F9E6ACD865387F227/DE736503824045B4BD228C35BA2E7B28] every.every/(1/1) ...SULTS.-1_'field2'_'field'
2016-05-18 02:07:15,346 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246]
2016-05-18 02:07:15,347 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:15,348 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 5EF3E809A62D46C69A57A2429F58345C on vertex vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246]
2016-05-18 02:07:15,350 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 5EF3E809A62D46C69A57A2429F58345C asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:15,354 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:15,354 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:15,354 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:15,354 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:15,355 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 5EF3E809A62D46C69A57A2429F58345C on vertex vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246]
2016-05-18 02:07:15,358 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:15,358 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {5EF3E809A62D46C69A57A2429F58345C=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:15,359 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246] parallelism set to 1
2016-05-18 02:07:15,359 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:15,360 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246], numAdditionalOutputs=1
2016-05-18 02:07:15,360 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=CE171CA76041455F892500AA085B9578, vertex=vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:15,361 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for A15449096F4640BA93ED31492B13D246:CE171CA76041455F892500AA085B9578 using oldmapred API
2016-05-18 02:07:15,362 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=CE171CA76041455F892500AA085B9578, vertexName=A15449096F4640BA93ED31492B13D246, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:15,362 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235155_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A15449096F4640BA93ED31492B13D246, vertexId=vertex_1463537235155_0001_1_00, initRequestedTime=1463537235345, initedTime=1463537235360, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:15,363 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246] with distanceFromRoot: 0
2016-05-18 02:07:15,363 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:15,364 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235155_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537235155_0001_1_00, startRequestedTime=1463537235347, startedTime=1463537235364
2016-05-18 02:07:15,364 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235155_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537235155_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:15,364 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:15,364 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in A15449096F4640BA93ED31492B13D246
2016-05-18 02:07:15,365 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246]
2016-05-18 02:07:15,365 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235155_0001_1][Event:TASK_STARTED]: vertexName=A15449096F4640BA93ED31492B13D246, taskId=task_1463537235155_0001_1_00_000000, scheduledTime=1463537235365, launchTime=1463537235365
2016-05-18 02:07:15,366 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246]
2016-05-18 02:07:15,367 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:15,368 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537235155_0001_00_000001, launchTime=1463537235368
2016-05-18 02:07:15,369 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537235155_0001_00_000001
2016-05-18 02:07:15,370 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537235155_0001_1_00_000000_0] started. Is using containerId: [container_1463537235155_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:15,370 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235155_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A15449096F4640BA93ED31492B13D246, taskAttemptId=attempt_1463537235155_0001_1_00_000000_0, startTime=1463537235370, containerId=container_1463537235155_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:15,371 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537235155_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537235155_0001_1_00_000000_0
2016-05-18 02:07:15 Starting to run new task attempt: attempt_1463537235155_0001_1_00_000000_0
2016-05-18 02:07:15,372 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537235155_0001_1_00_000000_0
2016-05-18 02:07:15,372 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:15,373 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:15,373 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [41C7E84CBB66407F9E6ACD865387F227/DE736503824045B4BD228C35BA2E7B28] every.every/(1/1) ...SULTS.-1_'field2'_'field', VertexName: A15449096F4640BA93ED31492B13D246, VertexParallelism: 1, TaskAttemptID:attempt_1463537235155_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5EF3E809A62D46C69A57A2429F58345C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CE171CA76041455F892500AA085B9578, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:15,373 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:15,374 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537235155_0001_1_00_000000_0
2016-05-18 02:07:15,381 INFO input.MRInput (MRInput.java:initialize(440)) - 5EF3E809A62D46C69A57A2429F58345C using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:15,381 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 5EF3E809A62D46C69A57A2429F58345C
2016-05-18 02:07:15,382 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537235155_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:15,383 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:15,383 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537235155_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:15,389 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537235155_0001, with dag id: dag_1463537235155_0001_1
2016-05-18 02:07:15,400 INFO output.MROutput (MROutput.java:initialize(421)) - CE171CA76041455F892500AA085B9578: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:15,401 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A15449096F4640BA93ED31492B13D246, ordinal: 0
2016-05-18 02:07:15,401 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A15449096F4640BA93ED31492B13D246, mem on start (mb), free: 289, total: 388, max: 672
2016-05-18 02:07:15,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:15,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:15,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:15,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:15,403 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[5EF3E809A62D46C69A57A2429F58345C:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [CE171CA76041455F892500AA085B9578:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:15,403 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:15,404 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:15,404 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:15,405 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[5EF3E809A62D46C69A57A2429F58345C:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [CE171CA76041455F892500AA085B9578:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:15,405 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:15,405 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 5EF3E809A62D46C69A57A2429F58345C being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:15,405 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 5EF3E809A62D46C69A57A2429F58345C
2016-05-18 02:07:15,406 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:15,407 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:15,407 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:15,407 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537235155_0001_1_00_000000_0
2016-05-18 02:07:15,410 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:15,410 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 5EF3E809A62D46C69A57A2429F58345C initialized RecordReader from event
2016-05-18 02:07:15,416 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:15,417 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 5EF3E809A62D46C69A57A2429F58345C
2016-05-18 02:07:15,418 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field'"], id: CE171CA76041455F892500AA085B9578
2016-05-18 02:07:15,418 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field'"]
2016-05-18 02:07:15,420 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:15,421 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A15449096F4640BA93ED31492B13D246, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:15,427 INFO output.MROutput (MROutput.java:close(526)) - CE171CA76041455F892500AA085B9578 closed
2016-05-18 02:07:15,428 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537235155_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:15,429 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537235155_0001_1_00_000000_0
2016-05-18 02:07:15,429 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A15449096F4640BA93ED31492B13D246, mem on close (mb), free: 287, total: 388, max: 672
2016-05-18 02:07:15,430 INFO output.MROutput (MROutput.java:close(526)) - CE171CA76041455F892500AA085B9578 closed
2016-05-18 02:07:15,431 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:15 Completed running task attempt: attempt_1463537235155_0001_1_00_000000_0
2016-05-18 02:07:15,432 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537235155_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:15,432 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537235155_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:15,433 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537235155_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537235422, Process_Duration=7, Process_End_Time=1463537235429, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:15,436 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537235155_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:15,437 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:15,437 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235155_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A15449096F4640BA93ED31492B13D246, taskAttemptId=attempt_1463537235155_0001_1_00_000000_0, creationTime=1463537235366, allocationTime=1463537235368, startTime=1463537235370, finishTime=1463537235436, timeTaken=66, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:15,437 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:15,438 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:15,438 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537235155_0001_1_00_000000_0
2016-05-18 02:07:15,438 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537235155_0001_00_000001
2016-05-18 02:07:15,439 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235155_0001_1][Event:TASK_FINISHED]: vertexName=A15449096F4640BA93ED31492B13D246, taskId=task_1463537235155_0001_1_00_000000, startTime=1463537235370, finishTime=1463537235438, timeTaken=68, status=SUCCEEDED, successfulAttemptID=attempt_1463537235155_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537235422, Process_Duration=7, Process_End_Time=1463537235429, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:15,440 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:15,440 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246]
2016-05-18 02:07:15,441 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235155_0001_1][Event:VERTEX_FINISHED]: vertexName=A15449096F4640BA93ED31492B13D246, vertexId=vertex_1463537235155_0001_1_00, initRequestedTime=1463537235345, initedTime=1463537235360, startRequestedTime=1463537235347, startedTime=1463537235364, finishTime=1463537235440, timeTaken=76, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537235422, Process_Duration=7, Process_End_Time=1463537235429, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537235370, firstTasksToStart=[ task_1463537235155_0001_1_00_000000 ], lastTaskFinishTime=1463537235436, lastTasksToFinish=[ task_1463537235155_0001_1_00_000000 ], minTaskDuration=66, maxTaskDuration=66, avgTaskDuration=66.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537235155_0001_1_00_000000 ], longestDurationTasks=[ task_1463537235155_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:15,442 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:15,442 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235155_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537235155_0001_00_000001, stoppedTime=1463537235442, exitStatus=0
2016-05-18 02:07:15,442 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:15,442 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:15,443 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537235155_0001_1
2016-05-18 02:07:15,445 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235155_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537235155_0001_1
2016-05-18 02:07:15,445 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: CE171CA76041455F892500AA085B9578 for vertex: vertex_1463537235155_0001_1_00 [A15449096F4640BA93ED31492B13D246], outputName: CE171CA76041455F892500AA085B9578
2016-05-18 02:07:15,445 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537235155_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:15,450 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537235155_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:15,451 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537235155_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:15,451 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537235155_0001_00_000001: 13 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:15,452 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537235155_0001_00_000001, Exiting
2016-05-18 02:07:15,452 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537235155_0001_00_000001
2016-05-18 02:07:15,452 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537235155_0001_00_000001
2016-05-18 02:07:15,453 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537235155_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:15,453 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537235155_0001_00_000001 completed successfully
2016-05-18 02:07:15,461 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:CE171CA76041455F892500AA085B9578 of vertex/vertexGroup:A15449096F4640BA93ED31492B13D246 isVertexGroupOutput:false
2016-05-18 02:07:15,461 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:15,463 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537235155_0001_1, queueSize=0
2016-05-18 02:07:15,463 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235155_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537235155_0001_1, startTime=1463537235344, finishTime=1463537235461, timeTaken=117, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=230, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537235422, Process_Duration=7, Process_End_Time=1463537235429, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:15,464 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537235155_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:15,464 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537235155_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:15 Completed Dag: dag_1463537235155_0001_1
2016-05-18 02:07:15,464 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537235155_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:15,465 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537235155_0001_1_post
2016-05-18 02:07:15,465 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:15,465 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:15,465 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:15,465 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[41C7E84CBB66407F9E6ACD865387F227/DE736503824045B4BD228C35BA2E7B28] every.every/(1/1) ...SULTS.-1_'field2'_'field', with id=dag_1463537235155_0001_1
2016-05-18 02:07:15,465 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[41C7E84CBB66407F9E6ACD865387F227/DE736503824045B4BD228C35BA2E7B28] every.every/(1/1) ...SULTS.-1_'field2'_'field', with id=dag_1463537235155_0001_1
2016-05-18 02:07:15,481 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...SULTS.-1_'field2'_'field', applicationId=application_1463537235155_0001
2016-05-18 02:07:15,486 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:15,486 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:15,486 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:15,487 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:15,487 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:15,488 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:15,488 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:15,489 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:15,489 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:15,489 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:15,490 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:15,491 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:15,492 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:15,495 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:15,500 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/41C7E84CBB66407F9E6ACD865387F227/DE736503824045B4BD228C35BA2E7B28/.tez/application_1463537235155_0001
2016-05-18 02:07:15,502 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field'/_temporary
2016-05-18 02:07:15,503 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.374
2016-05-18 02:07:15,594 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_'field2'_'field'
2016-05-18 02:07:15,626 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:15,627 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:15,686 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-05-18 02:07:15,699 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.035
2016-05-18 02:07:15,700 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:15,700 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:15,700 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:15,721 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:15,721 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_RESULTS
2016-05-18 02:07:15,735 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:15,736 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:15,736 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_RESULTS"]
2016-05-18 02:07:15,737 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:15,738 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:15,738 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:15,740 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...SULTS.-1_'field2'_RESULTS
2016-05-18 02:07:15,740 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:15,758 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:15,763 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:15,764 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:15,764 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:15,776 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/93545AE9E32749F8A75774BB49807015/F00DD9222F4848EB96381128059216BC/.tez/application_1463537235764_0001 doesn't exist and is created
2016-05-18 02:07:15,807 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:15,807 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:15,809 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/93545AE9E32749F8A75774BB49807015/F00DD9222F4848EB96381128059216BC/.tez/application_1463537235764_0001_wd
2016-05-18 02:07:15,831 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537235764_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:15,832 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:15,832 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:15,834 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:15,834 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:15,835 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:15,835 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:15,836 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:15,838 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:15,839 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:15,839 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:15,839 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:15,839 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:15,839 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:15,839 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:15,840 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:15,840 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:15,840 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:15,840 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:15,840 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:15,843 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:15,844 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:15,844 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:15,845 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/93545AE9E32749F8A75774BB49807015/F00DD9222F4848EB96381128059216BC/.tez/application_1463537235764_0001_wd/localmode-log-dir
2016-05-18 02:07:15,845 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/93545AE9E32749F8A75774BB49807015/F00DD9222F4848EB96381128059216BC/.tez/application_1463537235764_0001_wd/localmode-log-dir/history.txt.appattempt_1463537235764_0001_000000, maxErrors=10
2016-05-18 02:07:15,845 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/93545AE9E32749F8A75774BB49807015/F00DD9222F4848EB96381128059216BC/.tez/application_1463537235764_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:15,846 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537235764_0001_000000, appSubmitTime=1463537235830, launchTime=1463537235830
2016-05-18 02:07:15,853 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:15,860 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:36229
2016-05-18 02:07:15,861 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537235764
2016-05-18 02:07:15,863 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537235764_0001_000000, startTime=1463537235863
2016-05-18 02:07:15,863 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:15,907 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:15,907 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:15,908 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...SULTS.-1_'field2'_RESULTS, applicationId=application_1463537235764_0001, dagName=[93545AE9E32749F8A75774BB49807015/F00DD9222F4848EB96381128059216BC] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS
2016-05-18 02:07:15,936 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [93545AE9E32749F8A75774BB49807015/F00DD9222F4848EB96381128059216BC] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS
2016-05-18 02:07:15,937 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:15,938 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537235764_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/93545AE9E32749F8A75774BB49807015/F00DD9222F4848EB96381128059216BC/.tez/application_1463537235764_0001_wd/localmode-log-dir/dag_1463537235764_0001_1.dot
2016-05-18 02:07:15,939 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/93545AE9E32749F8A75774BB49807015/F00DD9222F4848EB96381128059216BC/.tez/application_1463537235764_0001_wd/localmode-log-dir/dag_1463537235764_0001_1-tez-dag.pb.txt
2016-05-18 02:07:15,946 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537235764_0001_1
2016-05-18 02:07:15,946 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:15,946 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [93545AE9E32749F8A75774BB49807015/F00DD9222F4848EB96381128059216BC] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS
2016-05-18 02:07:15 Running Dag: dag_1463537235764_0001_1
2016-05-18 02:07:15,962 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235764_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537235764_0001_1, submitTime=1463537235937
2016-05-18 02:07:15,964 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/93545AE9E32749F8A75774BB49807015/F00DD9222F4848EB96381128059216BC/.tez/application_1463537235764_0001/tez-conf.pb" } size: 40947 timestamp: 1463537235000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:15,964 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8]
2016-05-18 02:07:15,964 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8]
2016-05-18 02:07:15,965 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:15,965 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:15,967 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235764_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537235764_0001_1, initTime=1463537235966
2016-05-18 02:07:15,967 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537235764_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:15,967 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:15,968 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235764_0001_1][Event:DAG_STARTED]: dagID=dag_1463537235764_0001_1, startTime=1463537235968
2016-05-18 02:07:15,968 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537235764_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:15,968 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: BD039B2A7F684F8FA1420ABDC2E9EAB8 : {36DF0883B2B54D9A98BF4CFE616EDD1F={InputName=36DF0883B2B54D9A98BF4CFE616EDD1F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:15,968 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...SULTS.-1_'field2'_RESULTS, applicationId=application_1463537235764_0001, dagName=[93545AE9E32749F8A75774BB49807015/F00DD9222F4848EB96381128059216BC] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS
2016-05-18 02:07:15,969 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 36DF0883B2B54D9A98BF4CFE616EDD1F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:15,969 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8]
2016-05-18 02:07:15,969 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8]
2016-05-18 02:07:15,969 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8]
2016-05-18 02:07:15,970 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8]
2016-05-18 02:07:15,972 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:15,972 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 36DF0883B2B54D9A98BF4CFE616EDD1F on vertex vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8]
2016-05-18 02:07:15,975 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 36DF0883B2B54D9A98BF4CFE616EDD1F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:15,978 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:15,979 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:15,979 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:15,979 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:15,980 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 36DF0883B2B54D9A98BF4CFE616EDD1F on vertex vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8]
2016-05-18 02:07:15,981 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:15,982 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {36DF0883B2B54D9A98BF4CFE616EDD1F=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:15,982 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8] parallelism set to 1
2016-05-18 02:07:15,982 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:15,983 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8], numAdditionalOutputs=1
2016-05-18 02:07:15,984 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=17BAD961DCD242B0B44F386580267DD7, vertex=vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:15,986 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for BD039B2A7F684F8FA1420ABDC2E9EAB8:17BAD961DCD242B0B44F386580267DD7 using oldmapred API
2016-05-18 02:07:15,986 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=17BAD961DCD242B0B44F386580267DD7, vertexName=BD039B2A7F684F8FA1420ABDC2E9EAB8, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:15,988 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235764_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BD039B2A7F684F8FA1420ABDC2E9EAB8, vertexId=vertex_1463537235764_0001_1_00, initRequestedTime=1463537235968, initedTime=1463537235983, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:15,988 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8] with distanceFromRoot: 0
2016-05-18 02:07:15,988 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:15,990 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235764_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537235764_0001_1_00, startRequestedTime=1463537235972, startedTime=1463537235990
2016-05-18 02:07:15,991 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235764_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537235764_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:15,992 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:15,992 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in BD039B2A7F684F8FA1420ABDC2E9EAB8
2016-05-18 02:07:15,993 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8]
2016-05-18 02:07:15,994 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235764_0001_1][Event:TASK_STARTED]: vertexName=BD039B2A7F684F8FA1420ABDC2E9EAB8, taskId=task_1463537235764_0001_1_00_000000, scheduledTime=1463537235993, launchTime=1463537235993
2016-05-18 02:07:15,994 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8]
2016-05-18 02:07:15,995 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:15,998 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537235764_0001_00_000001, launchTime=1463537235997
2016-05-18 02:07:15,998 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537235764_0001_00_000001
2016-05-18 02:07:16,000 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537235764_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537235764_0001_1_00_000000_0
2016-05-18 02:07:16,000 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537235764_0001_1_00_000000_0] started. Is using containerId: [container_1463537235764_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:16,000 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235764_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BD039B2A7F684F8FA1420ABDC2E9EAB8, taskAttemptId=attempt_1463537235764_0001_1_00_000000_0, startTime=1463537235999, containerId=container_1463537235764_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:16,002 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537235764_0001, with dag id: dag_1463537235764_0001_1
2016-05-18 02:07:16 Starting to run new task attempt: attempt_1463537235764_0001_1_00_000000_0
2016-05-18 02:07:16,003 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537235764_0001_1_00_000000_0
2016-05-18 02:07:16,003 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:16,003 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:16,004 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [93545AE9E32749F8A75774BB49807015/F00DD9222F4848EB96381128059216BC] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS, VertexName: BD039B2A7F684F8FA1420ABDC2E9EAB8, VertexParallelism: 1, TaskAttemptID:attempt_1463537235764_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=36DF0883B2B54D9A98BF4CFE616EDD1F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=17BAD961DCD242B0B44F386580267DD7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:16,005 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:16,007 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537235764_0001_1_00_000000_0
2016-05-18 02:07:16,010 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537235764_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:16,011 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537235764_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:16,018 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:16,020 INFO input.MRInput (MRInput.java:initialize(440)) - 36DF0883B2B54D9A98BF4CFE616EDD1F using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:16,020 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 36DF0883B2B54D9A98BF4CFE616EDD1F
2016-05-18 02:07:16,030 INFO output.MROutput (MROutput.java:initialize(421)) - 17BAD961DCD242B0B44F386580267DD7: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:16,032 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BD039B2A7F684F8FA1420ABDC2E9EAB8, ordinal: 0
2016-05-18 02:07:16,032 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BD039B2A7F684F8FA1420ABDC2E9EAB8, mem on start (mb), free: 186, total: 388, max: 672
2016-05-18 02:07:16,032 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:16,033 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:16,033 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:16,033 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:16,034 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[36DF0883B2B54D9A98BF4CFE616EDD1F:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [17BAD961DCD242B0B44F386580267DD7:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:16,035 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:16,035 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:16,035 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:16,035 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[36DF0883B2B54D9A98BF4CFE616EDD1F:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [17BAD961DCD242B0B44F386580267DD7:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:16,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:16,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 36DF0883B2B54D9A98BF4CFE616EDD1F being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:16,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:16,037 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:16,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 36DF0883B2B54D9A98BF4CFE616EDD1F
2016-05-18 02:07:16,037 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:16,038 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537235764_0001_1_00_000000_0
2016-05-18 02:07:16,039 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:16,040 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 36DF0883B2B54D9A98BF4CFE616EDD1F initialized RecordReader from event
2016-05-18 02:07:16,044 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:16,045 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 36DF0883B2B54D9A98BF4CFE616EDD1F
2016-05-18 02:07:16,045 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_RESULTS"], id: 17BAD961DCD242B0B44F386580267DD7
2016-05-18 02:07:16,045 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_RESULTS"]
2016-05-18 02:07:16,046 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:16,046 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BD039B2A7F684F8FA1420ABDC2E9EAB8, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:16,050 INFO output.MROutput (MROutput.java:close(526)) - 17BAD961DCD242B0B44F386580267DD7 closed
2016-05-18 02:07:16,050 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537235764_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:16,052 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537235764_0001_1_00_000000_0
2016-05-18 02:07:16,052 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BD039B2A7F684F8FA1420ABDC2E9EAB8, mem on close (mb), free: 184, total: 388, max: 672
2016-05-18 02:07:16,052 INFO output.MROutput (MROutput.java:close(526)) - 17BAD961DCD242B0B44F386580267DD7 closed
2016-05-18 02:07:16,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:16 Completed running task attempt: attempt_1463537235764_0001_1_00_000000_0
2016-05-18 02:07:16,054 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537235764_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:16,054 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537235764_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:16,055 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537235764_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537236047, Process_Duration=4, Process_End_Time=1463537236051, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:16,056 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537235764_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:16,056 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:16,057 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:16,057 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:16,057 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235764_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BD039B2A7F684F8FA1420ABDC2E9EAB8, taskAttemptId=attempt_1463537235764_0001_1_00_000000_0, creationTime=1463537235994, allocationTime=1463537235996, startTime=1463537235999, finishTime=1463537236057, timeTaken=58, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:16,058 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537235764_0001_00_000001
2016-05-18 02:07:16,058 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537235764_0001_1_00_000000_0
2016-05-18 02:07:16,060 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235764_0001_1][Event:TASK_FINISHED]: vertexName=BD039B2A7F684F8FA1420ABDC2E9EAB8, taskId=task_1463537235764_0001_1_00_000000, startTime=1463537235999, finishTime=1463537236058, timeTaken=59, status=SUCCEEDED, successfulAttemptID=attempt_1463537235764_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537236047, Process_Duration=4, Process_End_Time=1463537236051, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:16,060 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:16,060 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8]
2016-05-18 02:07:16,062 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235764_0001_1][Event:VERTEX_FINISHED]: vertexName=BD039B2A7F684F8FA1420ABDC2E9EAB8, vertexId=vertex_1463537235764_0001_1_00, initRequestedTime=1463537235968, initedTime=1463537235983, startRequestedTime=1463537235972, startedTime=1463537235990, finishTime=1463537236061, timeTaken=71, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537236047, Process_Duration=4, Process_End_Time=1463537236051, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537235999, firstTasksToStart=[ task_1463537235764_0001_1_00_000000 ], lastTaskFinishTime=1463537236057, lastTasksToFinish=[ task_1463537235764_0001_1_00_000000 ], minTaskDuration=58, maxTaskDuration=58, avgTaskDuration=58.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537235764_0001_1_00_000000 ], longestDurationTasks=[ task_1463537235764_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:16,062 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:16,062 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235764_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537235764_0001_00_000001, stoppedTime=1463537236062, exitStatus=0
2016-05-18 02:07:16,062 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:16,063 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:16,063 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537235764_0001_1
2016-05-18 02:07:16,063 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235764_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537235764_0001_1
2016-05-18 02:07:16,063 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 17BAD961DCD242B0B44F386580267DD7 for vertex: vertex_1463537235764_0001_1_00 [BD039B2A7F684F8FA1420ABDC2E9EAB8], outputName: 17BAD961DCD242B0B44F386580267DD7
2016-05-18 02:07:16,063 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537235764_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:16,064 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537235764_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:16,070 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537235764_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:16,070 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537235764_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:16,071 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537235764_0001_00_000001, Exiting
2016-05-18 02:07:16,071 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537235764_0001_00_000001
2016-05-18 02:07:16,071 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537235764_0001_00_000001
2016-05-18 02:07:16,075 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537235764_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:16,076 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537235764_0001_00_000001 completed successfully
2016-05-18 02:07:16,077 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:17BAD961DCD242B0B44F386580267DD7 of vertex/vertexGroup:BD039B2A7F684F8FA1420ABDC2E9EAB8 isVertexGroupOutput:false
2016-05-18 02:07:16,078 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:16,080 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537235764_0001_1, queueSize=0
2016-05-18 02:07:16,080 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537235764_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537235764_0001_1, startTime=1463537235967, finishTime=1463537236078, timeTaken=111, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=190, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537236047, Process_Duration=4, Process_End_Time=1463537236051, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:16,080 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537235764_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:16,080 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537235764_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:16 Completed Dag: dag_1463537235764_0001_1
2016-05-18 02:07:16,081 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537235764_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:16,081 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537235764_0001_1_post
2016-05-18 02:07:16,081 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:16,081 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:16,081 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:16,081 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[93545AE9E32749F8A75774BB49807015/F00DD9222F4848EB96381128059216BC] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS, with id=dag_1463537235764_0001_1
2016-05-18 02:07:16,082 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[93545AE9E32749F8A75774BB49807015/F00DD9222F4848EB96381128059216BC] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS, with id=dag_1463537235764_0001_1
2016-05-18 02:07:16,088 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...SULTS.-1_'field2'_RESULTS, applicationId=application_1463537235764_0001
2016-05-18 02:07:16,091 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:16,092 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:16,092 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:16,092 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:16,092 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:16,092 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:16,094 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:16,094 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:16,094 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:16,094 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:16,094 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:16,095 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:16,095 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:16,096 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:16,100 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/93545AE9E32749F8A75774BB49807015/F00DD9222F4848EB96381128059216BC/.tez/application_1463537235764_0001
2016-05-18 02:07:16,100 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_RESULTS/_temporary
2016-05-18 02:07:16,101 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.365
2016-05-18 02:07:16,213 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_'field2'_RESULTS
2016-05-18 02:07:16,293 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:16,294 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:16,350 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-05-18 02:07:16,359 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-05-18 02:07:16,360 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:16,360 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:16,360 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:16,378 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:16,379 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_ALL
2016-05-18 02:07:16,390 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:16,390 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:16,390 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_ALL"]
2016-05-18 02:07:16,391 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:16,391 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:16,391 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:16,392 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...'_RESULTS.-1_'field2'_ALL
2016-05-18 02:07:16,392 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:16,411 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:16,415 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:16,416 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:16,416 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:16,426 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F45FE572CD9E4131BED3ED3F58485930/FCDA4BA7DE284AB18E670F6D652CD8F5/.tez/application_1463537236415_0001 doesn't exist and is created
2016-05-18 02:07:16,447 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:16,447 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:16,448 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F45FE572CD9E4131BED3ED3F58485930/FCDA4BA7DE284AB18E670F6D652CD8F5/.tez/application_1463537236415_0001_wd
2016-05-18 02:07:16,464 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537236415_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:16,464 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:16,464 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:16,465 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:16,465 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:16,465 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:16,466 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:16,466 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:16,468 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:16,468 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:16,469 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:16,469 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:16,469 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:16,469 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:16,469 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:16,470 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:16,470 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:16,470 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:16,470 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:16,470 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:16,472 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:16,473 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:16,473 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:16,474 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F45FE572CD9E4131BED3ED3F58485930/FCDA4BA7DE284AB18E670F6D652CD8F5/.tez/application_1463537236415_0001_wd/localmode-log-dir
2016-05-18 02:07:16,474 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F45FE572CD9E4131BED3ED3F58485930/FCDA4BA7DE284AB18E670F6D652CD8F5/.tez/application_1463537236415_0001_wd/localmode-log-dir/history.txt.appattempt_1463537236415_0001_000000, maxErrors=10
2016-05-18 02:07:16,474 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F45FE572CD9E4131BED3ED3F58485930/FCDA4BA7DE284AB18E670F6D652CD8F5/.tez/application_1463537236415_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:16,474 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537236415_0001_000000, appSubmitTime=1463537236463, launchTime=1463537236463
2016-05-18 02:07:16,484 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:16,487 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:58214
2016-05-18 02:07:16,488 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537236415
2016-05-18 02:07:16,490 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537236415_0001_000000, startTime=1463537236490
2016-05-18 02:07:16,490 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:16,547 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:16,547 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:16,548 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...'_RESULTS.-1_'field2'_ALL, applicationId=application_1463537236415_0001, dagName=[F45FE572CD9E4131BED3ED3F58485930/FCDA4BA7DE284AB18E670F6D652CD8F5] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL
2016-05-18 02:07:16,567 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [F45FE572CD9E4131BED3ED3F58485930/FCDA4BA7DE284AB18E670F6D652CD8F5] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL
2016-05-18 02:07:16,567 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:16,568 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537236415_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F45FE572CD9E4131BED3ED3F58485930/FCDA4BA7DE284AB18E670F6D652CD8F5/.tez/application_1463537236415_0001_wd/localmode-log-dir/dag_1463537236415_0001_1.dot
2016-05-18 02:07:16,569 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F45FE572CD9E4131BED3ED3F58485930/FCDA4BA7DE284AB18E670F6D652CD8F5/.tez/application_1463537236415_0001_wd/localmode-log-dir/dag_1463537236415_0001_1-tez-dag.pb.txt
2016-05-18 02:07:16,573 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537236415_0001_1
2016-05-18 02:07:16,573 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:16,573 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [F45FE572CD9E4131BED3ED3F58485930/FCDA4BA7DE284AB18E670F6D652CD8F5] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL
2016-05-18 02:07:16 Running Dag: dag_1463537236415_0001_1
2016-05-18 02:07:16,587 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236415_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537236415_0001_1, submitTime=1463537236567
2016-05-18 02:07:16,588 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F45FE572CD9E4131BED3ED3F58485930/FCDA4BA7DE284AB18E670F6D652CD8F5/.tez/application_1463537236415_0001/tez-conf.pb" } size: 40947 timestamp: 1463537236000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:16,588 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120]
2016-05-18 02:07:16,588 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120]
2016-05-18 02:07:16,589 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:16,589 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:16,589 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236415_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537236415_0001_1, initTime=1463537236589
2016-05-18 02:07:16,590 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537236415_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:16,591 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:16,592 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...'_RESULTS.-1_'field2'_ALL, applicationId=application_1463537236415_0001, dagName=[F45FE572CD9E4131BED3ED3F58485930/FCDA4BA7DE284AB18E670F6D652CD8F5] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL
2016-05-18 02:07:16,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236415_0001_1][Event:DAG_STARTED]: dagID=dag_1463537236415_0001_1, startTime=1463537236592
2016-05-18 02:07:16,592 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537236415_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:16,592 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: D5D396077F874CCA9899308A27C43120 : {0EE25CD814F945BA9B24EAEFC0D8D80B={InputName=0EE25CD814F945BA9B24EAEFC0D8D80B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:16,592 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 0EE25CD814F945BA9B24EAEFC0D8D80B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:16,592 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120]
2016-05-18 02:07:16,593 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120]
2016-05-18 02:07:16,593 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120]
2016-05-18 02:07:16,593 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120]
2016-05-18 02:07:16,594 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:16,594 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 0EE25CD814F945BA9B24EAEFC0D8D80B on vertex vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120]
2016-05-18 02:07:16,596 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 0EE25CD814F945BA9B24EAEFC0D8D80B asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:16,600 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:16,600 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:16,600 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:16,601 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:16,601 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 0EE25CD814F945BA9B24EAEFC0D8D80B on vertex vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120]
2016-05-18 02:07:16,602 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:16,603 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {0EE25CD814F945BA9B24EAEFC0D8D80B=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:16,603 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120] parallelism set to 1
2016-05-18 02:07:16,603 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:16,604 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120], numAdditionalOutputs=1
2016-05-18 02:07:16,604 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=D3EC1ED82D3C4E8894DB7AC602730DB5, vertex=vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:16,606 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D5D396077F874CCA9899308A27C43120:D3EC1ED82D3C4E8894DB7AC602730DB5 using oldmapred API
2016-05-18 02:07:16,606 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=D3EC1ED82D3C4E8894DB7AC602730DB5, vertexName=D5D396077F874CCA9899308A27C43120, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:16,607 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236415_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D5D396077F874CCA9899308A27C43120, vertexId=vertex_1463537236415_0001_1_00, initRequestedTime=1463537236592, initedTime=1463537236604, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:16,607 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120] with distanceFromRoot: 0
2016-05-18 02:07:16,609 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:16,610 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236415_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537236415_0001_1_00, startRequestedTime=1463537236594, startedTime=1463537236609
2016-05-18 02:07:16,611 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236415_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537236415_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:16,611 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:16,611 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in D5D396077F874CCA9899308A27C43120
2016-05-18 02:07:16,612 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120]
2016-05-18 02:07:16,613 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236415_0001_1][Event:TASK_STARTED]: vertexName=D5D396077F874CCA9899308A27C43120, taskId=task_1463537236415_0001_1_00_000000, scheduledTime=1463537236613, launchTime=1463537236613
2016-05-18 02:07:16,613 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120]
2016-05-18 02:07:16,614 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:16,617 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537236415_0001_00_000001, launchTime=1463537236617
2016-05-18 02:07:16,618 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537236415_0001_00_000001
2016-05-18 02:07:16,619 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537236415_0001_1_00_000000_0] started. Is using containerId: [container_1463537236415_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:16,620 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236415_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D5D396077F874CCA9899308A27C43120, taskAttemptId=attempt_1463537236415_0001_1_00_000000_0, startTime=1463537236619, containerId=container_1463537236415_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:16,621 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537236415_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537236415_0001_1_00_000000_0
2016-05-18 02:07:16 Starting to run new task attempt: attempt_1463537236415_0001_1_00_000000_0
2016-05-18 02:07:16,621 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537236415_0001_1_00_000000_0
2016-05-18 02:07:16,622 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:16,622 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:16,622 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [F45FE572CD9E4131BED3ED3F58485930/FCDA4BA7DE284AB18E670F6D652CD8F5] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL, VertexName: D5D396077F874CCA9899308A27C43120, VertexParallelism: 1, TaskAttemptID:attempt_1463537236415_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0EE25CD814F945BA9B24EAEFC0D8D80B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D3EC1ED82D3C4E8894DB7AC602730DB5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:16,622 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:16,626 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537236415_0001_1_00_000000_0
2016-05-18 02:07:16,627 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537236415_0001, with dag id: dag_1463537236415_0001_1
2016-05-18 02:07:16,629 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537236415_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:16,630 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537236415_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:16,637 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:16,638 INFO input.MRInput (MRInput.java:initialize(440)) - 0EE25CD814F945BA9B24EAEFC0D8D80B using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:16,639 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 0EE25CD814F945BA9B24EAEFC0D8D80B
2016-05-18 02:07:16,650 INFO output.MROutput (MROutput.java:initialize(421)) - D3EC1ED82D3C4E8894DB7AC602730DB5: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:16,651 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D5D396077F874CCA9899308A27C43120, ordinal: 0
2016-05-18 02:07:16,652 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5D396077F874CCA9899308A27C43120, mem on start (mb), free: 291, total: 382, max: 672
2016-05-18 02:07:16,652 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:16,652 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:16,653 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:16,653 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:16,654 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[0EE25CD814F945BA9B24EAEFC0D8D80B:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [D3EC1ED82D3C4E8894DB7AC602730DB5:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:16,654 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:16,655 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:16,656 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:16,656 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[0EE25CD814F945BA9B24EAEFC0D8D80B:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [D3EC1ED82D3C4E8894DB7AC602730DB5:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:16,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:16,657 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 0EE25CD814F945BA9B24EAEFC0D8D80B being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:16,657 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 0EE25CD814F945BA9B24EAEFC0D8D80B
2016-05-18 02:07:16,657 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:16,658 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:16,659 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:16,659 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537236415_0001_1_00_000000_0
2016-05-18 02:07:16,661 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:16,661 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 0EE25CD814F945BA9B24EAEFC0D8D80B initialized RecordReader from event
2016-05-18 02:07:16,663 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:16,664 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 0EE25CD814F945BA9B24EAEFC0D8D80B
2016-05-18 02:07:16,665 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_ALL"], id: D3EC1ED82D3C4E8894DB7AC602730DB5
2016-05-18 02:07:16,665 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_ALL"]
2016-05-18 02:07:16,666 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:16,666 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D5D396077F874CCA9899308A27C43120, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:16,670 INFO output.MROutput (MROutput.java:close(526)) - D3EC1ED82D3C4E8894DB7AC602730DB5 closed
2016-05-18 02:07:16,671 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537236415_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:16,673 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537236415_0001_1_00_000000_0
2016-05-18 02:07:16,673 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5D396077F874CCA9899308A27C43120, mem on close (mb), free: 289, total: 382, max: 672
2016-05-18 02:07:16,673 INFO output.MROutput (MROutput.java:close(526)) - D3EC1ED82D3C4E8894DB7AC602730DB5 closed
2016-05-18 02:07:16,673 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:16 Completed running task attempt: attempt_1463537236415_0001_1_00_000000_0
2016-05-18 02:07:16,675 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537236415_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:16,675 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537236415_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:16,675 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537236415_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537236667, Process_Duration=5, Process_End_Time=1463537236672, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:16,676 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537236415_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:16,677 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:16,677 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:16,677 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:16,677 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537236415_0001_00_000001
2016-05-18 02:07:16,678 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236415_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D5D396077F874CCA9899308A27C43120, taskAttemptId=attempt_1463537236415_0001_1_00_000000_0, creationTime=1463537236613, allocationTime=1463537236616, startTime=1463537236619, finishTime=1463537236678, timeTaken=59, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:16,679 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537236415_0001_1_00_000000_0
2016-05-18 02:07:16,679 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236415_0001_1][Event:TASK_FINISHED]: vertexName=D5D396077F874CCA9899308A27C43120, taskId=task_1463537236415_0001_1_00_000000, startTime=1463537236619, finishTime=1463537236679, timeTaken=60, status=SUCCEEDED, successfulAttemptID=attempt_1463537236415_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537236667, Process_Duration=5, Process_End_Time=1463537236672, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:16,680 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:16,680 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120]
2016-05-18 02:07:16,682 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236415_0001_1][Event:VERTEX_FINISHED]: vertexName=D5D396077F874CCA9899308A27C43120, vertexId=vertex_1463537236415_0001_1_00, initRequestedTime=1463537236592, initedTime=1463537236604, startRequestedTime=1463537236594, startedTime=1463537236609, finishTime=1463537236680, timeTaken=71, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537236667, Process_Duration=5, Process_End_Time=1463537236672, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537236619, firstTasksToStart=[ task_1463537236415_0001_1_00_000000 ], lastTaskFinishTime=1463537236678, lastTasksToFinish=[ task_1463537236415_0001_1_00_000000 ], minTaskDuration=59, maxTaskDuration=59, avgTaskDuration=59.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537236415_0001_1_00_000000 ], longestDurationTasks=[ task_1463537236415_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:16,682 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:16,682 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236415_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537236415_0001_00_000001, stoppedTime=1463537236682, exitStatus=0
2016-05-18 02:07:16,682 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:16,682 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:16,683 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537236415_0001_1
2016-05-18 02:07:16,683 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236415_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537236415_0001_1
2016-05-18 02:07:16,684 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: D3EC1ED82D3C4E8894DB7AC602730DB5 for vertex: vertex_1463537236415_0001_1_00 [D5D396077F874CCA9899308A27C43120], outputName: D3EC1ED82D3C4E8894DB7AC602730DB5
2016-05-18 02:07:16,688 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537236415_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:16,688 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537236415_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:16,688 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537236415_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:16,689 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537236415_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:16,690 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537236415_0001_00_000001, Exiting
2016-05-18 02:07:16,690 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537236415_0001_00_000001
2016-05-18 02:07:16,690 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537236415_0001_00_000001
2016-05-18 02:07:16,693 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537236415_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:16,694 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537236415_0001_00_000001 completed successfully
2016-05-18 02:07:16,698 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:D3EC1ED82D3C4E8894DB7AC602730DB5 of vertex/vertexGroup:D5D396077F874CCA9899308A27C43120 isVertexGroupOutput:false
2016-05-18 02:07:16,698 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:16,700 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537236415_0001_1, queueSize=0
2016-05-18 02:07:16,700 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236415_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537236415_0001_1, startTime=1463537236591, finishTime=1463537236698, timeTaken=107, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=260, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537236667, Process_Duration=5, Process_End_Time=1463537236672, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:16,700 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537236415_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:16,700 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537236415_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:16 Completed Dag: dag_1463537236415_0001_1
2016-05-18 02:07:16,700 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537236415_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:16,701 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537236415_0001_1_post
2016-05-18 02:07:16,701 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:16,701 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:16,701 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:16,701 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[F45FE572CD9E4131BED3ED3F58485930/FCDA4BA7DE284AB18E670F6D652CD8F5] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL, with id=dag_1463537236415_0001_1
2016-05-18 02:07:16,701 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[F45FE572CD9E4131BED3ED3F58485930/FCDA4BA7DE284AB18E670F6D652CD8F5] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL, with id=dag_1463537236415_0001_1
2016-05-18 02:07:16,708 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...'_RESULTS.-1_'field2'_ALL, applicationId=application_1463537236415_0001
2016-05-18 02:07:16,710 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:16,710 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:16,711 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:16,711 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:16,711 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:16,711 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:16,712 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:16,712 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:16,712 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:16,713 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:16,713 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:16,714 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:16,714 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:16,715 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:16,718 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F45FE572CD9E4131BED3ED3F58485930/FCDA4BA7DE284AB18E670F6D652CD8F5/.tez/application_1463537236415_0001
2016-05-18 02:07:16,719 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_ALL/_temporary
2016-05-18 02:07:16,720 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.329
2016-05-18 02:07:16,825 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_'field2'_ALL
2016-05-18 02:07:16,854 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:16,854 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:16,910 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-18 02:07:16,922 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.029
2016-05-18 02:07:16,922 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:16,923 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:16,923 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:16,941 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:16,941 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_-1
2016-05-18 02:07:16,954 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:16,954 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:16,954 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_-1"]
2016-05-18 02:07:16,954 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:16,954 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:16,954 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:16,955 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ld'_RESULTS.-1_UNKNOWN_-1
2016-05-18 02:07:16,955 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:16,971 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:16,976 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:16,977 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:16,977 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:16,986 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/5C422EF9FFD5461EBA18023809CE0AA8/F4AEEB116702454A83D5505BBB3C7FDC/.tez/application_1463537236976_0001 doesn't exist and is created
2016-05-18 02:07:17,007 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:17,007 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:17,008 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/5C422EF9FFD5461EBA18023809CE0AA8/F4AEEB116702454A83D5505BBB3C7FDC/.tez/application_1463537236976_0001_wd
2016-05-18 02:07:17,024 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537236976_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:17,025 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:17,025 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:17,028 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:17,028 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:17,028 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:17,028 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:17,030 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:17,033 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:17,033 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:17,034 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:17,034 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:17,034 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:17,034 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:17,035 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:17,035 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:17,035 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:17,035 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:17,036 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:17,036 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:17,037 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:17,038 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:17,039 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:17,039 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/5C422EF9FFD5461EBA18023809CE0AA8/F4AEEB116702454A83D5505BBB3C7FDC/.tez/application_1463537236976_0001_wd/localmode-log-dir
2016-05-18 02:07:17,040 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/5C422EF9FFD5461EBA18023809CE0AA8/F4AEEB116702454A83D5505BBB3C7FDC/.tez/application_1463537236976_0001_wd/localmode-log-dir/history.txt.appattempt_1463537236976_0001_000000, maxErrors=10
2016-05-18 02:07:17,040 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/5C422EF9FFD5461EBA18023809CE0AA8/F4AEEB116702454A83D5505BBB3C7FDC/.tez/application_1463537236976_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:17,040 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537236976_0001_000000, appSubmitTime=1463537237023, launchTime=1463537237023
2016-05-18 02:07:17,049 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:17,057 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:53792
2016-05-18 02:07:17,057 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537236976
2016-05-18 02:07:17,059 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537236976_0001_000000, startTime=1463537237059
2016-05-18 02:07:17,059 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:17,107 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:17,108 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:17,108 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1, applicationId=application_1463537236976_0001, dagName=[5C422EF9FFD5461EBA18023809CE0AA8/F4AEEB116702454A83D5505BBB3C7FDC] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1
2016-05-18 02:07:17,132 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [5C422EF9FFD5461EBA18023809CE0AA8/F4AEEB116702454A83D5505BBB3C7FDC] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1
2016-05-18 02:07:17,133 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:17,134 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537236976_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/5C422EF9FFD5461EBA18023809CE0AA8/F4AEEB116702454A83D5505BBB3C7FDC/.tez/application_1463537236976_0001_wd/localmode-log-dir/dag_1463537236976_0001_1.dot
2016-05-18 02:07:17,135 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/5C422EF9FFD5461EBA18023809CE0AA8/F4AEEB116702454A83D5505BBB3C7FDC/.tez/application_1463537236976_0001_wd/localmode-log-dir/dag_1463537236976_0001_1-tez-dag.pb.txt
2016-05-18 02:07:17,142 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537236976_0001_1
2016-05-18 02:07:17,142 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:17,142 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [5C422EF9FFD5461EBA18023809CE0AA8/F4AEEB116702454A83D5505BBB3C7FDC] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1
2016-05-18 02:07:17 Running Dag: dag_1463537236976_0001_1
2016-05-18 02:07:17,158 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236976_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537236976_0001_1, submitTime=1463537237132
2016-05-18 02:07:17,159 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/5C422EF9FFD5461EBA18023809CE0AA8/F4AEEB116702454A83D5505BBB3C7FDC/.tez/application_1463537236976_0001/tez-conf.pb" } size: 40947 timestamp: 1463537236000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:17,160 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971]
2016-05-18 02:07:17,160 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971]
2016-05-18 02:07:17,160 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:17,161 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:17,161 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236976_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537236976_0001_1, initTime=1463537237161
2016-05-18 02:07:17,163 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537236976_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:17,163 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:17,165 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236976_0001_1][Event:DAG_STARTED]: dagID=dag_1463537236976_0001_1, startTime=1463537237165
2016-05-18 02:07:17,165 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1, applicationId=application_1463537236976_0001, dagName=[5C422EF9FFD5461EBA18023809CE0AA8/F4AEEB116702454A83D5505BBB3C7FDC] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1
2016-05-18 02:07:17,165 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537236976_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:17,166 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: B6CE1AB0823845E8AFA19ACAC1BA6971 : {2B3C11EAC54C47CDA2CCACF21E1BF3C1={InputName=2B3C11EAC54C47CDA2CCACF21E1BF3C1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:17,166 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 2B3C11EAC54C47CDA2CCACF21E1BF3C1, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:17,166 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971]
2016-05-18 02:07:17,166 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971]
2016-05-18 02:07:17,166 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971]
2016-05-18 02:07:17,166 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971]
2016-05-18 02:07:17,167 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:17,168 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 2B3C11EAC54C47CDA2CCACF21E1BF3C1 on vertex vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971]
2016-05-18 02:07:17,171 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 2B3C11EAC54C47CDA2CCACF21E1BF3C1 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:17,173 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:17,173 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:17,174 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:17,174 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:17,175 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 2B3C11EAC54C47CDA2CCACF21E1BF3C1 on vertex vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971]
2016-05-18 02:07:17,176 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:17,176 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {2B3C11EAC54C47CDA2CCACF21E1BF3C1=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:17,178 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971] parallelism set to 1
2016-05-18 02:07:17,178 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:17,179 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971], numAdditionalOutputs=1
2016-05-18 02:07:17,179 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=75E9662E7BA247E19B7F4863F391846B, vertex=vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:17,181 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for B6CE1AB0823845E8AFA19ACAC1BA6971:75E9662E7BA247E19B7F4863F391846B using oldmapred API
2016-05-18 02:07:17,181 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=75E9662E7BA247E19B7F4863F391846B, vertexName=B6CE1AB0823845E8AFA19ACAC1BA6971, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:17,182 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236976_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B6CE1AB0823845E8AFA19ACAC1BA6971, vertexId=vertex_1463537236976_0001_1_00, initRequestedTime=1463537237166, initedTime=1463537237179, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:17,183 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971] with distanceFromRoot: 0
2016-05-18 02:07:17,184 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:17,184 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236976_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537236976_0001_1_00, startRequestedTime=1463537237167, startedTime=1463537237184
2016-05-18 02:07:17,185 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236976_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537236976_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:17,185 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:17,185 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in B6CE1AB0823845E8AFA19ACAC1BA6971
2016-05-18 02:07:17,186 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971]
2016-05-18 02:07:17,186 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236976_0001_1][Event:TASK_STARTED]: vertexName=B6CE1AB0823845E8AFA19ACAC1BA6971, taskId=task_1463537236976_0001_1_00_000000, scheduledTime=1463537237186, launchTime=1463537237186
2016-05-18 02:07:17,187 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971]
2016-05-18 02:07:17,188 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:17,190 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537236976_0001_00_000001, launchTime=1463537237190
2016-05-18 02:07:17,191 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537236976_0001_00_000001
2016-05-18 02:07:17,192 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537236976_0001, with dag id: dag_1463537236976_0001_1
2016-05-18 02:07:17,192 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537236976_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537236976_0001_1_00_000000_0
2016-05-18 02:07:17,192 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537236976_0001_1_00_000000_0] started. Is using containerId: [container_1463537236976_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:17,192 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236976_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B6CE1AB0823845E8AFA19ACAC1BA6971, taskAttemptId=attempt_1463537236976_0001_1_00_000000_0, startTime=1463537237192, containerId=container_1463537236976_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:17 Starting to run new task attempt: attempt_1463537236976_0001_1_00_000000_0
2016-05-18 02:07:17,193 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537236976_0001_1_00_000000_0
2016-05-18 02:07:17,193 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:17,193 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:17,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5C422EF9FFD5461EBA18023809CE0AA8/F4AEEB116702454A83D5505BBB3C7FDC] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1, VertexName: B6CE1AB0823845E8AFA19ACAC1BA6971, VertexParallelism: 1, TaskAttemptID:attempt_1463537236976_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2B3C11EAC54C47CDA2CCACF21E1BF3C1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=75E9662E7BA247E19B7F4863F391846B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:17,195 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:17,198 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537236976_0001_1_00_000000_0
2016-05-18 02:07:17,201 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537236976_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:17,201 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537236976_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:17,205 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:17,208 INFO input.MRInput (MRInput.java:initialize(440)) - 2B3C11EAC54C47CDA2CCACF21E1BF3C1 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:17,208 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 2B3C11EAC54C47CDA2CCACF21E1BF3C1
2016-05-18 02:07:17,216 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B6CE1AB0823845E8AFA19ACAC1BA6971, ordinal: 0
2016-05-18 02:07:17,216 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B6CE1AB0823845E8AFA19ACAC1BA6971, mem on start (mb), free: 193, total: 382, max: 672
2016-05-18 02:07:17,216 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:17,216 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:17,217 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:17,220 INFO output.MROutput (MROutput.java:initialize(421)) - 75E9662E7BA247E19B7F4863F391846B: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:17,220 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:17,220 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[75E9662E7BA247E19B7F4863F391846B:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [2B3C11EAC54C47CDA2CCACF21E1BF3C1:INPUT:0:org.apache.tez.mapreduce.input.MRInput]
2016-05-18 02:07:17,220 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:17,221 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:17,221 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:17,221 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[75E9662E7BA247E19B7F4863F391846B:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [2B3C11EAC54C47CDA2CCACF21E1BF3C1:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0]
2016-05-18 02:07:17,221 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:17,222 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 2B3C11EAC54C47CDA2CCACF21E1BF3C1 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:17,222 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 2B3C11EAC54C47CDA2CCACF21E1BF3C1
2016-05-18 02:07:17,222 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:17,223 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:17,223 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:17,223 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537236976_0001_1_00_000000_0
2016-05-18 02:07:17,224 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:17,225 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 2B3C11EAC54C47CDA2CCACF21E1BF3C1 initialized RecordReader from event
2016-05-18 02:07:17,228 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:17,228 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 2B3C11EAC54C47CDA2CCACF21E1BF3C1
2016-05-18 02:07:17,229 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_-1"], id: 75E9662E7BA247E19B7F4863F391846B
2016-05-18 02:07:17,231 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_-1"]
2016-05-18 02:07:17,231 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:17,232 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B6CE1AB0823845E8AFA19ACAC1BA6971, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:17,235 INFO output.MROutput (MROutput.java:close(526)) - 75E9662E7BA247E19B7F4863F391846B closed
2016-05-18 02:07:17,236 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537236976_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:17,237 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537236976_0001_1_00_000000_0
2016-05-18 02:07:17,237 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B6CE1AB0823845E8AFA19ACAC1BA6971, mem on close (mb), free: 190, total: 382, max: 672
2016-05-18 02:07:17,237 INFO output.MROutput (MROutput.java:close(526)) - 75E9662E7BA247E19B7F4863F391846B closed
2016-05-18 02:07:17,237 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:17 Completed running task attempt: attempt_1463537236976_0001_1_00_000000_0
2016-05-18 02:07:17,238 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537236976_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:17,238 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537236976_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:17,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537236976_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537237232, Process_Duration=5, Process_End_Time=1463537237237, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:17,240 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537236976_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:17,240 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:17,240 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236976_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B6CE1AB0823845E8AFA19ACAC1BA6971, taskAttemptId=attempt_1463537236976_0001_1_00_000000_0, creationTime=1463537237187, allocationTime=1463537237189, startTime=1463537237192, finishTime=1463537237240, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:17,240 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:17,241 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537236976_0001_1_00_000000_0
2016-05-18 02:07:17,242 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:17,242 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236976_0001_1][Event:TASK_FINISHED]: vertexName=B6CE1AB0823845E8AFA19ACAC1BA6971, taskId=task_1463537236976_0001_1_00_000000, startTime=1463537237192, finishTime=1463537237242, timeTaken=50, status=SUCCEEDED, successfulAttemptID=attempt_1463537236976_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537237232, Process_Duration=5, Process_End_Time=1463537237237, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:17,242 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537236976_0001_00_000001
2016-05-18 02:07:17,243 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:17,243 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971]
2016-05-18 02:07:17,244 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236976_0001_1][Event:VERTEX_FINISHED]: vertexName=B6CE1AB0823845E8AFA19ACAC1BA6971, vertexId=vertex_1463537236976_0001_1_00, initRequestedTime=1463537237166, initedTime=1463537237179, startRequestedTime=1463537237167, startedTime=1463537237184, finishTime=1463537237243, timeTaken=59, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537237232, Process_Duration=5, Process_End_Time=1463537237237, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537237192, firstTasksToStart=[ task_1463537236976_0001_1_00_000000 ], lastTaskFinishTime=1463537237240, lastTasksToFinish=[ task_1463537236976_0001_1_00_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537236976_0001_1_00_000000 ], longestDurationTasks=[ task_1463537236976_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:17,244 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:17,245 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236976_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537236976_0001_00_000001, stoppedTime=1463537237244, exitStatus=0
2016-05-18 02:07:17,245 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:17,245 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:17,245 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537236976_0001_1
2016-05-18 02:07:17,245 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236976_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537236976_0001_1
2016-05-18 02:07:17,246 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 75E9662E7BA247E19B7F4863F391846B for vertex: vertex_1463537236976_0001_1_00 [B6CE1AB0823845E8AFA19ACAC1BA6971], outputName: 75E9662E7BA247E19B7F4863F391846B
2016-05-18 02:07:17,248 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537236976_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:17,248 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537236976_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:17,253 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537236976_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:17,253 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537236976_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:17,253 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537236976_0001_00_000001, Exiting
2016-05-18 02:07:17,254 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537236976_0001_00_000001
2016-05-18 02:07:17,254 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537236976_0001_00_000001
2016-05-18 02:07:17,254 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537236976_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:17,255 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537236976_0001_00_000001 completed successfully
2016-05-18 02:07:17,257 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:75E9662E7BA247E19B7F4863F391846B of vertex/vertexGroup:B6CE1AB0823845E8AFA19ACAC1BA6971 isVertexGroupOutput:false
2016-05-18 02:07:17,257 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:17,258 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537236976_0001_1, queueSize=0
2016-05-18 02:07:17,259 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537236976_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537236976_0001_1, startTime=1463537237163, finishTime=1463537237257, timeTaken=94, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=170, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537237232, Process_Duration=5, Process_End_Time=1463537237237, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:17,259 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537236976_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:17,259 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537236976_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:17 Completed Dag: dag_1463537236976_0001_1
2016-05-18 02:07:17,259 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537236976_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:17,259 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537236976_0001_1_post
2016-05-18 02:07:17,259 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:17,259 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:17,259 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:17,259 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[5C422EF9FFD5461EBA18023809CE0AA8/F4AEEB116702454A83D5505BBB3C7FDC] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1, with id=dag_1463537236976_0001_1
2016-05-18 02:07:17,260 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[5C422EF9FFD5461EBA18023809CE0AA8/F4AEEB116702454A83D5505BBB3C7FDC] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1, with id=dag_1463537236976_0001_1
2016-05-18 02:07:17,270 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1, applicationId=application_1463537236976_0001
2016-05-18 02:07:17,274 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:17,274 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:17,274 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:17,274 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:17,274 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:17,274 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:17,275 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:17,275 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:17,275 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:17,275 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:17,275 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:17,276 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:17,277 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:17,277 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:17,281 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/5C422EF9FFD5461EBA18023809CE0AA8/F4AEEB116702454A83D5505BBB3C7FDC/.tez/application_1463537236976_0001
2016-05-18 02:07:17,282 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_-1/_temporary
2016-05-18 02:07:17,283 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.327
2016-05-18 02:07:17,378 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_UNKNOWN_-1
2016-05-18 02:07:17,404 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:17,406 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:17,450 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.008
2016-05-18 02:07:17,453 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.011
2016-05-18 02:07:17,453 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 02:07:17,453 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 02:07:17,454 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_UNKNOWN_'field2'
2016-05-18 02:07:17,508 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:17,509 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:17,557 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-18 02:07:17,564 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-05-18 02:07:17,565 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:17,565 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:17,565 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:17,582 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:17,582 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_'field'
2016-05-18 02:07:17,595 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:17,596 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:17,596 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_'field'"]
2016-05-18 02:07:17,596 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:17,596 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:17,596 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:17,597 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ESULTS.-1_UNKNOWN_'field'
2016-05-18 02:07:17,597 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:17,611 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:17,616 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:17,616 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:17,616 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:17,625 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/62E113D45C7F45A58014AEB89CB31047/C4E58F4E52E54BCA93E756767CDE7393/.tez/application_1463537237616_0001 doesn't exist and is created
2016-05-18 02:07:17,645 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:17,645 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:17,647 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/62E113D45C7F45A58014AEB89CB31047/C4E58F4E52E54BCA93E756767CDE7393/.tez/application_1463537237616_0001_wd
2016-05-18 02:07:17,664 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537237616_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:17,664 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:17,664 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:17,666 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:17,666 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:17,667 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:17,667 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:17,668 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:17,670 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:17,670 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:17,670 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:17,671 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:17,671 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:17,671 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:17,671 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:17,671 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:17,671 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:17,671 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:17,672 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:17,672 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:17,673 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:17,674 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:17,674 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:17,674 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/62E113D45C7F45A58014AEB89CB31047/C4E58F4E52E54BCA93E756767CDE7393/.tez/application_1463537237616_0001_wd/localmode-log-dir
2016-05-18 02:07:17,675 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/62E113D45C7F45A58014AEB89CB31047/C4E58F4E52E54BCA93E756767CDE7393/.tez/application_1463537237616_0001_wd/localmode-log-dir/history.txt.appattempt_1463537237616_0001_000000, maxErrors=10
2016-05-18 02:07:17,675 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/62E113D45C7F45A58014AEB89CB31047/C4E58F4E52E54BCA93E756767CDE7393/.tez/application_1463537237616_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:17,675 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537237616_0001_000000, appSubmitTime=1463537237663, launchTime=1463537237663
2016-05-18 02:07:17,680 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:17,685 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:39363
2016-05-18 02:07:17,685 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537237616
2016-05-18 02:07:17,687 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537237616_0001_000000, startTime=1463537237687
2016-05-18 02:07:17,687 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:17,745 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:17,746 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:17,746 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_'field', applicationId=application_1463537237616_0001, dagName=[62E113D45C7F45A58014AEB89CB31047/C4E58F4E52E54BCA93E756767CDE7393] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field'
2016-05-18 02:07:17,767 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [62E113D45C7F45A58014AEB89CB31047/C4E58F4E52E54BCA93E756767CDE7393] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field'
2016-05-18 02:07:17,767 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:17,768 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537237616_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/62E113D45C7F45A58014AEB89CB31047/C4E58F4E52E54BCA93E756767CDE7393/.tez/application_1463537237616_0001_wd/localmode-log-dir/dag_1463537237616_0001_1.dot
2016-05-18 02:07:17,769 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/62E113D45C7F45A58014AEB89CB31047/C4E58F4E52E54BCA93E756767CDE7393/.tez/application_1463537237616_0001_wd/localmode-log-dir/dag_1463537237616_0001_1-tez-dag.pb.txt
2016-05-18 02:07:17,774 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537237616_0001_1
2016-05-18 02:07:17,774 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:17,775 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [62E113D45C7F45A58014AEB89CB31047/C4E58F4E52E54BCA93E756767CDE7393] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field'
2016-05-18 02:07:17 Running Dag: dag_1463537237616_0001_1
2016-05-18 02:07:17,791 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537237616_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537237616_0001_1, submitTime=1463537237767
2016-05-18 02:07:17,793 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/62E113D45C7F45A58014AEB89CB31047/C4E58F4E52E54BCA93E756767CDE7393/.tez/application_1463537237616_0001/tez-conf.pb" } size: 40947 timestamp: 1463537237000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:17,793 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118]
2016-05-18 02:07:17,793 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118]
2016-05-18 02:07:17,793 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:17,793 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:17,794 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537237616_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537237616_0001_1, initTime=1463537237793
2016-05-18 02:07:17,794 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537237616_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:17,795 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:17,795 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_'field', applicationId=application_1463537237616_0001, dagName=[62E113D45C7F45A58014AEB89CB31047/C4E58F4E52E54BCA93E756767CDE7393] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field'
2016-05-18 02:07:17,795 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537237616_0001_1][Event:DAG_STARTED]: dagID=dag_1463537237616_0001_1, startTime=1463537237795
2016-05-18 02:07:17,795 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537237616_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:17,796 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: C2073FBEF956413AB7D352E9719DE118 : {5ADB08CE2C964E48AB3A0260BA0D3CEE={InputName=5ADB08CE2C964E48AB3A0260BA0D3CEE}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:17,796 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 5ADB08CE2C964E48AB3A0260BA0D3CEE, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:17,796 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118]
2016-05-18 02:07:17,796 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118]
2016-05-18 02:07:17,796 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118]
2016-05-18 02:07:17,796 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118]
2016-05-18 02:07:17,797 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:17,797 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 5ADB08CE2C964E48AB3A0260BA0D3CEE on vertex vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118]
2016-05-18 02:07:17,800 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 5ADB08CE2C964E48AB3A0260BA0D3CEE asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:17,802 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:17,803 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:17,803 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:17,803 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:17,804 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 5ADB08CE2C964E48AB3A0260BA0D3CEE on vertex vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118]
2016-05-18 02:07:17,805 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:17,806 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {5ADB08CE2C964E48AB3A0260BA0D3CEE=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:17,806 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118] parallelism set to 1
2016-05-18 02:07:17,807 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:17,807 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118], numAdditionalOutputs=1
2016-05-18 02:07:17,807 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=3CEF54FD4B694E1F8326F41B903B767E, vertex=vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:17,810 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for C2073FBEF956413AB7D352E9719DE118:3CEF54FD4B694E1F8326F41B903B767E using oldmapred API
2016-05-18 02:07:17,810 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=3CEF54FD4B694E1F8326F41B903B767E, vertexName=C2073FBEF956413AB7D352E9719DE118, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:17,812 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537237616_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C2073FBEF956413AB7D352E9719DE118, vertexId=vertex_1463537237616_0001_1_00, initRequestedTime=1463537237796, initedTime=1463537237807, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:17,814 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118] with distanceFromRoot: 0
2016-05-18 02:07:17,814 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:17,814 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537237616_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537237616_0001_1_00, startRequestedTime=1463537237797, startedTime=1463537237814
2016-05-18 02:07:17,815 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537237616_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537237616_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:17,815 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:17,815 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in C2073FBEF956413AB7D352E9719DE118
2016-05-18 02:07:17,815 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118]
2016-05-18 02:07:17,818 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537237616_0001_1][Event:TASK_STARTED]: vertexName=C2073FBEF956413AB7D352E9719DE118, taskId=task_1463537237616_0001_1_00_000000, scheduledTime=1463537237818, launchTime=1463537237818
2016-05-18 02:07:17,818 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118]
2016-05-18 02:07:17,820 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537237616_0001, with dag id: dag_1463537237616_0001_1
2016-05-18 02:07:17,820 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:17,824 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537237616_0001_00_000001, launchTime=1463537237822
2016-05-18 02:07:17,825 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537237616_0001_00_000001
2016-05-18 02:07:17,826 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537237616_0001_1_00_000000_0] started. Is using containerId: [container_1463537237616_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:17,826 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537237616_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C2073FBEF956413AB7D352E9719DE118, taskAttemptId=attempt_1463537237616_0001_1_00_000000_0, startTime=1463537237825, containerId=container_1463537237616_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:17,828 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537237616_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537237616_0001_1_00_000000_0
2016-05-18 02:07:17 Starting to run new task attempt: attempt_1463537237616_0001_1_00_000000_0
2016-05-18 02:07:17,829 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537237616_0001_1_00_000000_0
2016-05-18 02:07:17,829 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:17,829 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:17,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [62E113D45C7F45A58014AEB89CB31047/C4E58F4E52E54BCA93E756767CDE7393] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field', VertexName: C2073FBEF956413AB7D352E9719DE118, VertexParallelism: 1, TaskAttemptID:attempt_1463537237616_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5ADB08CE2C964E48AB3A0260BA0D3CEE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=3CEF54FD4B694E1F8326F41B903B767E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:17,830 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:17,830 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537237616_0001_1_00_000000_0
2016-05-18 02:07:17,834 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537237616_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:17,834 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537237616_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:17,839 INFO input.MRInput (MRInput.java:initialize(440)) - 5ADB08CE2C964E48AB3A0260BA0D3CEE using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:17,840 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 5ADB08CE2C964E48AB3A0260BA0D3CEE
2016-05-18 02:07:17,840 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:17,851 INFO output.MROutput (MROutput.java:initialize(421)) - 3CEF54FD4B694E1F8326F41B903B767E: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:17,852 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C2073FBEF956413AB7D352E9719DE118, ordinal: 0
2016-05-18 02:07:17,852 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C2073FBEF956413AB7D352E9719DE118, mem on start (mb), free: 290, total: 385, max: 672
2016-05-18 02:07:17,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:17,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:17,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:17,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:17,854 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[3CEF54FD4B694E1F8326F41B903B767E:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [5ADB08CE2C964E48AB3A0260BA0D3CEE:INPUT:0:org.apache.tez.mapreduce.input.MRInput]
2016-05-18 02:07:17,854 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:17,854 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:17,855 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:17,855 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[3CEF54FD4B694E1F8326F41B903B767E:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [5ADB08CE2C964E48AB3A0260BA0D3CEE:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0]
2016-05-18 02:07:17,856 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:17,856 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 5ADB08CE2C964E48AB3A0260BA0D3CEE
2016-05-18 02:07:17,856 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 5ADB08CE2C964E48AB3A0260BA0D3CEE being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:17,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:17,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:17,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:17,858 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537237616_0001_1_00_000000_0
2016-05-18 02:07:17,859 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:17,859 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 5ADB08CE2C964E48AB3A0260BA0D3CEE initialized RecordReader from event
2016-05-18 02:07:17,862 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:17,862 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 5ADB08CE2C964E48AB3A0260BA0D3CEE
2016-05-18 02:07:17,863 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_'field'"], id: 3CEF54FD4B694E1F8326F41B903B767E
2016-05-18 02:07:17,863 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_'field'"]
2016-05-18 02:07:17,863 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:17,864 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C2073FBEF956413AB7D352E9719DE118, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:17,867 INFO output.MROutput (MROutput.java:close(526)) - 3CEF54FD4B694E1F8326F41B903B767E closed
2016-05-18 02:07:17,868 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537237616_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:17,869 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537237616_0001_1_00_000000_0
2016-05-18 02:07:17,869 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C2073FBEF956413AB7D352E9719DE118, mem on close (mb), free: 288, total: 385, max: 672
2016-05-18 02:07:17,869 INFO output.MROutput (MROutput.java:close(526)) - 3CEF54FD4B694E1F8326F41B903B767E closed
2016-05-18 02:07:17,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:17 Completed running task attempt: attempt_1463537237616_0001_1_00_000000_0
2016-05-18 02:07:17,871 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537237616_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:17,871 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537237616_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:17,871 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537237616_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537237864, Process_Duration=5, Process_End_Time=1463537237869, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:17,872 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537237616_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:17,872 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:17,872 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:17,872 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:17,873 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537237616_0001_00_000001
2016-05-18 02:07:17,873 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537237616_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C2073FBEF956413AB7D352E9719DE118, taskAttemptId=attempt_1463537237616_0001_1_00_000000_0, creationTime=1463537237818, allocationTime=1463537237822, startTime=1463537237825, finishTime=1463537237872, timeTaken=47, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:17,874 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537237616_0001_1_00_000000_0
2016-05-18 02:07:17,875 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537237616_0001_1][Event:TASK_FINISHED]: vertexName=C2073FBEF956413AB7D352E9719DE118, taskId=task_1463537237616_0001_1_00_000000, startTime=1463537237825, finishTime=1463537237874, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1463537237616_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537237864, Process_Duration=5, Process_End_Time=1463537237869, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:17,875 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:17,875 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118]
2016-05-18 02:07:17,877 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537237616_0001_1][Event:VERTEX_FINISHED]: vertexName=C2073FBEF956413AB7D352E9719DE118, vertexId=vertex_1463537237616_0001_1_00, initRequestedTime=1463537237796, initedTime=1463537237807, startRequestedTime=1463537237797, startedTime=1463537237814, finishTime=1463537237875, timeTaken=61, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537237864, Process_Duration=5, Process_End_Time=1463537237869, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537237825, firstTasksToStart=[ task_1463537237616_0001_1_00_000000 ], lastTaskFinishTime=1463537237872, lastTasksToFinish=[ task_1463537237616_0001_1_00_000000 ], minTaskDuration=47, maxTaskDuration=47, avgTaskDuration=47.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537237616_0001_1_00_000000 ], longestDurationTasks=[ task_1463537237616_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:17,877 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:17,877 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537237616_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537237616_0001_00_000001, stoppedTime=1463537237877, exitStatus=0
2016-05-18 02:07:17,877 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:17,877 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:17,878 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537237616_0001_1
2016-05-18 02:07:17,878 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537237616_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537237616_0001_1
2016-05-18 02:07:17,878 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 3CEF54FD4B694E1F8326F41B903B767E for vertex: vertex_1463537237616_0001_1_00 [C2073FBEF956413AB7D352E9719DE118], outputName: 3CEF54FD4B694E1F8326F41B903B767E
2016-05-18 02:07:17,880 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537237616_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:17,880 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537237616_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:17,884 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537237616_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:17,884 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537237616_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:17,884 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537237616_0001_00_000001, Exiting
2016-05-18 02:07:17,884 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537237616_0001_00_000001
2016-05-18 02:07:17,884 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537237616_0001_00_000001
2016-05-18 02:07:17,886 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537237616_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:17,886 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537237616_0001_00_000001 completed successfully
2016-05-18 02:07:17,888 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:3CEF54FD4B694E1F8326F41B903B767E of vertex/vertexGroup:C2073FBEF956413AB7D352E9719DE118 isVertexGroupOutput:false
2016-05-18 02:07:17,889 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:17,890 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537237616_0001_1, queueSize=0
2016-05-18 02:07:17,891 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537237616_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537237616_0001_1, startTime=1463537237795, finishTime=1463537237889, timeTaken=94, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=160, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537237864, Process_Duration=5, Process_End_Time=1463537237869, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:17,891 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537237616_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:17,891 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537237616_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:17 Completed Dag: dag_1463537237616_0001_1
2016-05-18 02:07:17,891 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537237616_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:17,891 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537237616_0001_1_post
2016-05-18 02:07:17,891 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:17,891 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:17,892 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:17,892 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[62E113D45C7F45A58014AEB89CB31047/C4E58F4E52E54BCA93E756767CDE7393] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field', with id=dag_1463537237616_0001_1
2016-05-18 02:07:17,892 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[62E113D45C7F45A58014AEB89CB31047/C4E58F4E52E54BCA93E756767CDE7393] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field', with id=dag_1463537237616_0001_1
2016-05-18 02:07:17,899 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_'field', applicationId=application_1463537237616_0001
2016-05-18 02:07:17,902 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:17,902 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:17,902 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:17,902 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:17,902 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:17,903 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:17,904 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:17,904 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:17,904 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:17,905 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:17,905 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:17,905 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:17,905 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:17,905 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:17,909 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/62E113D45C7F45A58014AEB89CB31047/C4E58F4E52E54BCA93E756767CDE7393/.tez/application_1463537237616_0001
2016-05-18 02:07:17,909 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_'field'/_temporary
2016-05-18 02:07:17,910 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.314
2016-05-18 02:07:17,986 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_UNKNOWN_'field'
2016-05-18 02:07:18,013 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:18,013 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:18,060 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-18 02:07:18,068 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-05-18 02:07:18,068 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:18,068 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:18,069 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:18,085 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:18,086 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_RESULTS
2016-05-18 02:07:18,097 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:18,097 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:18,097 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_RESULTS"]
2016-05-18 02:07:18,097 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:18,098 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:18,098 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:18,098 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ESULTS.-1_UNKNOWN_RESULTS
2016-05-18 02:07:18,099 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:18,115 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:18,119 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:18,120 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:18,120 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:18,129 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F7C6B8A090684AD8B8BAD0A457E0DDBC/CE00AEB4EDA04447B875C1D92E3E5A11/.tez/application_1463537238120_0001 doesn't exist and is created
2016-05-18 02:07:18,150 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:18,150 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:18,151 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F7C6B8A090684AD8B8BAD0A457E0DDBC/CE00AEB4EDA04447B875C1D92E3E5A11/.tez/application_1463537238120_0001_wd
2016-05-18 02:07:18,168 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537238120_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:18,168 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:18,168 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:18,171 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:18,171 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:18,171 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:18,171 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:18,172 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:18,173 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:18,173 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:18,173 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:18,173 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:18,174 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:18,174 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:18,174 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:18,174 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:18,175 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:18,175 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:18,175 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:18,175 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:18,177 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:18,178 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:18,178 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:18,178 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F7C6B8A090684AD8B8BAD0A457E0DDBC/CE00AEB4EDA04447B875C1D92E3E5A11/.tez/application_1463537238120_0001_wd/localmode-log-dir
2016-05-18 02:07:18,179 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F7C6B8A090684AD8B8BAD0A457E0DDBC/CE00AEB4EDA04447B875C1D92E3E5A11/.tez/application_1463537238120_0001_wd/localmode-log-dir/history.txt.appattempt_1463537238120_0001_000000, maxErrors=10
2016-05-18 02:07:18,179 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F7C6B8A090684AD8B8BAD0A457E0DDBC/CE00AEB4EDA04447B875C1D92E3E5A11/.tez/application_1463537238120_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:18,179 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537238120_0001_000000, appSubmitTime=1463537238167, launchTime=1463537238167
2016-05-18 02:07:18,186 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:18,188 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:38268
2016-05-18 02:07:18,189 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537238120
2016-05-18 02:07:18,191 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537238120_0001_000000, startTime=1463537238191
2016-05-18 02:07:18,191 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:18,250 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:18,250 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:18,251 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_RESULTS, applicationId=application_1463537238120_0001, dagName=[F7C6B8A090684AD8B8BAD0A457E0DDBC/CE00AEB4EDA04447B875C1D92E3E5A11] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS
2016-05-18 02:07:18,268 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [F7C6B8A090684AD8B8BAD0A457E0DDBC/CE00AEB4EDA04447B875C1D92E3E5A11] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS
2016-05-18 02:07:18,269 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:18,270 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537238120_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F7C6B8A090684AD8B8BAD0A457E0DDBC/CE00AEB4EDA04447B875C1D92E3E5A11/.tez/application_1463537238120_0001_wd/localmode-log-dir/dag_1463537238120_0001_1.dot
2016-05-18 02:07:18,271 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F7C6B8A090684AD8B8BAD0A457E0DDBC/CE00AEB4EDA04447B875C1D92E3E5A11/.tez/application_1463537238120_0001_wd/localmode-log-dir/dag_1463537238120_0001_1-tez-dag.pb.txt
2016-05-18 02:07:18,281 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537238120_0001_1
2016-05-18 02:07:18,281 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:18,282 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [F7C6B8A090684AD8B8BAD0A457E0DDBC/CE00AEB4EDA04447B875C1D92E3E5A11] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS
2016-05-18 02:07:18 Running Dag: dag_1463537238120_0001_1
2016-05-18 02:07:18,298 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238120_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537238120_0001_1, submitTime=1463537238269
2016-05-18 02:07:18,299 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F7C6B8A090684AD8B8BAD0A457E0DDBC/CE00AEB4EDA04447B875C1D92E3E5A11/.tez/application_1463537238120_0001/tez-conf.pb" } size: 40947 timestamp: 1463537238000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:18,299 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D]
2016-05-18 02:07:18,299 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D]
2016-05-18 02:07:18,300 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:18,300 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:18,300 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238120_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537238120_0001_1, initTime=1463537238300
2016-05-18 02:07:18,300 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537238120_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:18,302 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:18,303 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238120_0001_1][Event:DAG_STARTED]: dagID=dag_1463537238120_0001_1, startTime=1463537238302
2016-05-18 02:07:18,303 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537238120_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:18,303 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: EF53CEBAC2B349C0A2F2D9A905C4672D : {B1642945A5DB4FD498EF7041CDF34DFF={InputName=B1642945A5DB4FD498EF7041CDF34DFF}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:18,303 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: B1642945A5DB4FD498EF7041CDF34DFF, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:18,303 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D]
2016-05-18 02:07:18,303 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D]
2016-05-18 02:07:18,303 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D]
2016-05-18 02:07:18,303 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D]
2016-05-18 02:07:18,304 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:18,304 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: B1642945A5DB4FD498EF7041CDF34DFF on vertex vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D]
2016-05-18 02:07:18,305 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_RESULTS, applicationId=application_1463537238120_0001, dagName=[F7C6B8A090684AD8B8BAD0A457E0DDBC/CE00AEB4EDA04447B875C1D92E3E5A11] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS
2016-05-18 02:07:18,307 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input B1642945A5DB4FD498EF7041CDF34DFF asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:18,311 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:18,311 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:18,312 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:18,312 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:18,312 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: B1642945A5DB4FD498EF7041CDF34DFF on vertex vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D]
2016-05-18 02:07:18,314 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:18,315 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {B1642945A5DB4FD498EF7041CDF34DFF=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:18,316 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D] parallelism set to 1
2016-05-18 02:07:18,316 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:18,316 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D], numAdditionalOutputs=1
2016-05-18 02:07:18,316 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=C1C4CD3B68AE440897E115D21CA17CD2, vertex=vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:18,319 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for EF53CEBAC2B349C0A2F2D9A905C4672D:C1C4CD3B68AE440897E115D21CA17CD2 using oldmapred API
2016-05-18 02:07:18,319 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=C1C4CD3B68AE440897E115D21CA17CD2, vertexName=EF53CEBAC2B349C0A2F2D9A905C4672D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:18,320 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238120_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EF53CEBAC2B349C0A2F2D9A905C4672D, vertexId=vertex_1463537238120_0001_1_00, initRequestedTime=1463537238303, initedTime=1463537238316, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:18,321 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D] with distanceFromRoot: 0
2016-05-18 02:07:18,321 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:18,322 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238120_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537238120_0001_1_00, startRequestedTime=1463537238304, startedTime=1463537238321
2016-05-18 02:07:18,322 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238120_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537238120_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:18,322 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:18,323 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in EF53CEBAC2B349C0A2F2D9A905C4672D
2016-05-18 02:07:18,323 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D]
2016-05-18 02:07:18,324 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238120_0001_1][Event:TASK_STARTED]: vertexName=EF53CEBAC2B349C0A2F2D9A905C4672D, taskId=task_1463537238120_0001_1_00_000000, scheduledTime=1463537238324, launchTime=1463537238324
2016-05-18 02:07:18,324 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D]
2016-05-18 02:07:18,325 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:18,329 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537238120_0001_00_000001, launchTime=1463537238329
2016-05-18 02:07:18,330 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537238120_0001_00_000001
2016-05-18 02:07:18,332 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537238120_0001_1_00_000000_0] started. Is using containerId: [container_1463537238120_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:18,332 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238120_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EF53CEBAC2B349C0A2F2D9A905C4672D, taskAttemptId=attempt_1463537238120_0001_1_00_000000_0, startTime=1463537238331, containerId=container_1463537238120_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:18,333 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537238120_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537238120_0001_1_00_000000_0
2016-05-18 02:07:18 Starting to run new task attempt: attempt_1463537238120_0001_1_00_000000_0
2016-05-18 02:07:18,334 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537238120_0001_1_00_000000_0
2016-05-18 02:07:18,335 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:18,335 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:18,335 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [F7C6B8A090684AD8B8BAD0A457E0DDBC/CE00AEB4EDA04447B875C1D92E3E5A11] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS, VertexName: EF53CEBAC2B349C0A2F2D9A905C4672D, VertexParallelism: 1, TaskAttemptID:attempt_1463537238120_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B1642945A5DB4FD498EF7041CDF34DFF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C1C4CD3B68AE440897E115D21CA17CD2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:18,336 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:18,338 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537238120_0001_1_00_000000_0
2016-05-18 02:07:18,340 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537238120_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:18,340 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537238120_0001, with dag id: dag_1463537238120_0001_1
2016-05-18 02:07:18,341 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537238120_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:18,352 INFO input.MRInput (MRInput.java:initialize(440)) - B1642945A5DB4FD498EF7041CDF34DFF using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:18,352 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: B1642945A5DB4FD498EF7041CDF34DFF
2016-05-18 02:07:18,354 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:18,362 INFO output.MROutput (MROutput.java:initialize(421)) - C1C4CD3B68AE440897E115D21CA17CD2: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:18,364 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EF53CEBAC2B349C0A2F2D9A905C4672D, ordinal: 0
2016-05-18 02:07:18,364 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EF53CEBAC2B349C0A2F2D9A905C4672D, mem on start (mb), free: 194, total: 385, max: 672
2016-05-18 02:07:18,364 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:18,364 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:18,364 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:18,365 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:18,365 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[C1C4CD3B68AE440897E115D21CA17CD2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [B1642945A5DB4FD498EF7041CDF34DFF:INPUT:0:org.apache.tez.mapreduce.input.MRInput]
2016-05-18 02:07:18,365 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:18,365 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:18,365 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:18,365 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[C1C4CD3B68AE440897E115D21CA17CD2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [B1642945A5DB4FD498EF7041CDF34DFF:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0]
2016-05-18 02:07:18,366 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:18,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: B1642945A5DB4FD498EF7041CDF34DFF
2016-05-18 02:07:18,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: B1642945A5DB4FD498EF7041CDF34DFF being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:18,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:18,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:18,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:18,368 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537238120_0001_1_00_000000_0
2016-05-18 02:07:18,370 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:18,370 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - B1642945A5DB4FD498EF7041CDF34DFF initialized RecordReader from event
2016-05-18 02:07:18,371 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:18,372 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: B1642945A5DB4FD498EF7041CDF34DFF
2016-05-18 02:07:18,372 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_RESULTS"], id: C1C4CD3B68AE440897E115D21CA17CD2
2016-05-18 02:07:18,372 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_RESULTS"]
2016-05-18 02:07:18,372 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:18,373 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EF53CEBAC2B349C0A2F2D9A905C4672D, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:18,375 INFO output.MROutput (MROutput.java:close(526)) - C1C4CD3B68AE440897E115D21CA17CD2 closed
2016-05-18 02:07:18,376 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537238120_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:18,377 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537238120_0001_1_00_000000_0
2016-05-18 02:07:18,377 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EF53CEBAC2B349C0A2F2D9A905C4672D, mem on close (mb), free: 193, total: 385, max: 672
2016-05-18 02:07:18,378 INFO output.MROutput (MROutput.java:close(526)) - C1C4CD3B68AE440897E115D21CA17CD2 closed
2016-05-18 02:07:18,378 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:18 Completed running task attempt: attempt_1463537238120_0001_1_00_000000_0
2016-05-18 02:07:18,379 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537238120_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:18,379 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537238120_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:18,380 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537238120_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537238373, Process_Duration=4, Process_End_Time=1463537238377, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:18,381 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537238120_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:18,381 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:18,381 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:18,381 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:18,381 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238120_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EF53CEBAC2B349C0A2F2D9A905C4672D, taskAttemptId=attempt_1463537238120_0001_1_00_000000_0, creationTime=1463537238324, allocationTime=1463537238328, startTime=1463537238331, finishTime=1463537238381, timeTaken=50, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:18,382 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537238120_0001_00_000001
2016-05-18 02:07:18,383 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537238120_0001_1_00_000000_0
2016-05-18 02:07:18,383 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238120_0001_1][Event:TASK_FINISHED]: vertexName=EF53CEBAC2B349C0A2F2D9A905C4672D, taskId=task_1463537238120_0001_1_00_000000, startTime=1463537238331, finishTime=1463537238383, timeTaken=52, status=SUCCEEDED, successfulAttemptID=attempt_1463537238120_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537238373, Process_Duration=4, Process_End_Time=1463537238377, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:18,384 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:18,384 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D]
2016-05-18 02:07:18,386 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238120_0001_1][Event:VERTEX_FINISHED]: vertexName=EF53CEBAC2B349C0A2F2D9A905C4672D, vertexId=vertex_1463537238120_0001_1_00, initRequestedTime=1463537238303, initedTime=1463537238316, startRequestedTime=1463537238304, startedTime=1463537238321, finishTime=1463537238384, timeTaken=63, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537238373, Process_Duration=4, Process_End_Time=1463537238377, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537238331, firstTasksToStart=[ task_1463537238120_0001_1_00_000000 ], lastTaskFinishTime=1463537238381, lastTasksToFinish=[ task_1463537238120_0001_1_00_000000 ], minTaskDuration=50, maxTaskDuration=50, avgTaskDuration=50.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537238120_0001_1_00_000000 ], longestDurationTasks=[ task_1463537238120_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:18,386 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:18,387 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238120_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537238120_0001_00_000001, stoppedTime=1463537238387, exitStatus=0
2016-05-18 02:07:18,387 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:18,387 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:18,387 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537238120_0001_1
2016-05-18 02:07:18,387 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238120_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537238120_0001_1
2016-05-18 02:07:18,387 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537238120_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:18,387 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: C1C4CD3B68AE440897E115D21CA17CD2 for vertex: vertex_1463537238120_0001_1_00 [EF53CEBAC2B349C0A2F2D9A905C4672D], outputName: C1C4CD3B68AE440897E115D21CA17CD2
2016-05-18 02:07:18,388 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537238120_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:18,393 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537238120_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:18,393 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537238120_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:18,393 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537238120_0001_00_000001, Exiting
2016-05-18 02:07:18,393 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537238120_0001_00_000001
2016-05-18 02:07:18,393 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537238120_0001_00_000001
2016-05-18 02:07:18,394 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537238120_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:18,394 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537238120_0001_00_000001 completed successfully
2016-05-18 02:07:18,398 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:C1C4CD3B68AE440897E115D21CA17CD2 of vertex/vertexGroup:EF53CEBAC2B349C0A2F2D9A905C4672D isVertexGroupOutput:false
2016-05-18 02:07:18,398 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:18,399 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537238120_0001_1, queueSize=0
2016-05-18 02:07:18,400 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238120_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537238120_0001_1, startTime=1463537238302, finishTime=1463537238398, timeTaken=96, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=180, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537238373, Process_Duration=4, Process_End_Time=1463537238377, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:18,400 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537238120_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:18,400 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537238120_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:18 Completed Dag: dag_1463537238120_0001_1
2016-05-18 02:07:18,400 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537238120_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:18,400 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537238120_0001_1_post
2016-05-18 02:07:18,401 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:18,401 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:18,401 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:18,401 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[F7C6B8A090684AD8B8BAD0A457E0DDBC/CE00AEB4EDA04447B875C1D92E3E5A11] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS, with id=dag_1463537238120_0001_1
2016-05-18 02:07:18,401 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[F7C6B8A090684AD8B8BAD0A457E0DDBC/CE00AEB4EDA04447B875C1D92E3E5A11] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS, with id=dag_1463537238120_0001_1
2016-05-18 02:07:18,408 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_RESULTS, applicationId=application_1463537238120_0001
2016-05-18 02:07:18,411 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:18,411 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:18,411 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:18,411 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:18,411 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:18,412 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:18,415 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:18,415 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:18,416 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:18,416 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:18,416 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:18,416 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:18,417 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:18,417 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:18,421 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F7C6B8A090684AD8B8BAD0A457E0DDBC/CE00AEB4EDA04447B875C1D92E3E5A11/.tez/application_1463537238120_0001
2016-05-18 02:07:18,421 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_RESULTS/_temporary
2016-05-18 02:07:18,422 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.324
2016-05-18 02:07:18,504 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_UNKNOWN_RESULTS
2016-05-18 02:07:18,533 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:18,533 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:18,580 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-18 02:07:18,585 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-05-18 02:07:18,585 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:18,585 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:18,586 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:18,620 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:18,621 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_ALL
2016-05-18 02:07:18,632 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:18,633 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:18,633 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_ALL"]
2016-05-18 02:07:18,633 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:18,633 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:18,633 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:18,633 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...d'_RESULTS.-1_UNKNOWN_ALL
2016-05-18 02:07:18,634 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:18,652 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:18,656 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:18,657 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:18,657 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:18,668 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/918E320DC0424733A237447706EFAC31/CAE15CA3E02A4DE48FAC6625C2C0AFEF/.tez/application_1463537238657_0001 doesn't exist and is created
2016-05-18 02:07:18,688 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:18,689 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:18,689 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/918E320DC0424733A237447706EFAC31/CAE15CA3E02A4DE48FAC6625C2C0AFEF/.tez/application_1463537238657_0001_wd
2016-05-18 02:07:18,705 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537238657_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:18,705 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:18,705 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:18,707 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:18,707 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:18,707 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:18,707 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:18,708 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:18,709 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:18,709 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:18,709 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:18,709 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:18,709 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:18,710 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:18,710 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:18,710 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:18,710 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:18,710 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:18,711 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:18,711 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:18,712 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:18,713 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:18,713 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:18,714 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/918E320DC0424733A237447706EFAC31/CAE15CA3E02A4DE48FAC6625C2C0AFEF/.tez/application_1463537238657_0001_wd/localmode-log-dir
2016-05-18 02:07:18,714 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/918E320DC0424733A237447706EFAC31/CAE15CA3E02A4DE48FAC6625C2C0AFEF/.tez/application_1463537238657_0001_wd/localmode-log-dir/history.txt.appattempt_1463537238657_0001_000000, maxErrors=10
2016-05-18 02:07:18,714 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/918E320DC0424733A237447706EFAC31/CAE15CA3E02A4DE48FAC6625C2C0AFEF/.tez/application_1463537238657_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:18,715 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537238657_0001_000000, appSubmitTime=1463537238705, launchTime=1463537238705
2016-05-18 02:07:18,719 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:18,725 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49323
2016-05-18 02:07:18,725 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537238657
2016-05-18 02:07:18,727 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537238657_0001_000000, startTime=1463537238727
2016-05-18 02:07:18,727 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:18,789 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:18,789 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:18,790 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL, applicationId=application_1463537238657_0001, dagName=[918E320DC0424733A237447706EFAC31/CAE15CA3E02A4DE48FAC6625C2C0AFEF] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL
2016-05-18 02:07:18,808 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [918E320DC0424733A237447706EFAC31/CAE15CA3E02A4DE48FAC6625C2C0AFEF] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL
2016-05-18 02:07:18,809 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:18,810 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537238657_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/918E320DC0424733A237447706EFAC31/CAE15CA3E02A4DE48FAC6625C2C0AFEF/.tez/application_1463537238657_0001_wd/localmode-log-dir/dag_1463537238657_0001_1.dot
2016-05-18 02:07:18,810 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/918E320DC0424733A237447706EFAC31/CAE15CA3E02A4DE48FAC6625C2C0AFEF/.tez/application_1463537238657_0001_wd/localmode-log-dir/dag_1463537238657_0001_1-tez-dag.pb.txt
2016-05-18 02:07:18,820 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537238657_0001_1
2016-05-18 02:07:18,820 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:18,820 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [918E320DC0424733A237447706EFAC31/CAE15CA3E02A4DE48FAC6625C2C0AFEF] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL
2016-05-18 02:07:18 Running Dag: dag_1463537238657_0001_1
2016-05-18 02:07:18,837 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238657_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537238657_0001_1, submitTime=1463537238808
2016-05-18 02:07:18,838 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/918E320DC0424733A237447706EFAC31/CAE15CA3E02A4DE48FAC6625C2C0AFEF/.tez/application_1463537238657_0001/tez-conf.pb" } size: 40947 timestamp: 1463537238000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:18,839 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA]
2016-05-18 02:07:18,839 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA]
2016-05-18 02:07:18,839 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:18,839 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:18,840 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238657_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537238657_0001_1, initTime=1463537238839
2016-05-18 02:07:18,840 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537238657_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:18,840 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:18,841 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238657_0001_1][Event:DAG_STARTED]: dagID=dag_1463537238657_0001_1, startTime=1463537238841
2016-05-18 02:07:18,841 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537238657_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:18,841 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: D82F27793E91497D96F6A5726D2934EA : {DC879B622DF64C2592E6A0F3E9F814AF={InputName=DC879B622DF64C2592E6A0F3E9F814AF}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:18,841 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: DC879B622DF64C2592E6A0F3E9F814AF, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:18,841 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA]
2016-05-18 02:07:18,842 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL, applicationId=application_1463537238657_0001, dagName=[918E320DC0424733A237447706EFAC31/CAE15CA3E02A4DE48FAC6625C2C0AFEF] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL
2016-05-18 02:07:18,842 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA]
2016-05-18 02:07:18,842 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA]
2016-05-18 02:07:18,842 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA]
2016-05-18 02:07:18,843 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:18,843 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: DC879B622DF64C2592E6A0F3E9F814AF on vertex vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA]
2016-05-18 02:07:18,845 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input DC879B622DF64C2592E6A0F3E9F814AF asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:18,847 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:18,847 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:18,847 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:18,848 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:18,848 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: DC879B622DF64C2592E6A0F3E9F814AF on vertex vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA]
2016-05-18 02:07:18,849 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:18,849 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {DC879B622DF64C2592E6A0F3E9F814AF=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:18,850 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA] parallelism set to 1
2016-05-18 02:07:18,850 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:18,851 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA], numAdditionalOutputs=1
2016-05-18 02:07:18,851 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=A50FB30328C6408D8DEE0C1E4E9449AD, vertex=vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:18,853 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D82F27793E91497D96F6A5726D2934EA:A50FB30328C6408D8DEE0C1E4E9449AD using oldmapred API
2016-05-18 02:07:18,853 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=A50FB30328C6408D8DEE0C1E4E9449AD, vertexName=D82F27793E91497D96F6A5726D2934EA, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:18,854 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238657_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D82F27793E91497D96F6A5726D2934EA, vertexId=vertex_1463537238657_0001_1_00, initRequestedTime=1463537238841, initedTime=1463537238851, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:18,854 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA] with distanceFromRoot: 0
2016-05-18 02:07:18,854 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:18,854 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238657_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537238657_0001_1_00, startRequestedTime=1463537238843, startedTime=1463537238854
2016-05-18 02:07:18,855 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238657_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537238657_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:18,855 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:18,855 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in D82F27793E91497D96F6A5726D2934EA
2016-05-18 02:07:18,856 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA]
2016-05-18 02:07:18,856 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238657_0001_1][Event:TASK_STARTED]: vertexName=D82F27793E91497D96F6A5726D2934EA, taskId=task_1463537238657_0001_1_00_000000, scheduledTime=1463537238856, launchTime=1463537238856
2016-05-18 02:07:18,857 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA]
2016-05-18 02:07:18,858 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:18,859 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537238657_0001_00_000001, launchTime=1463537238859
2016-05-18 02:07:18,860 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537238657_0001_00_000001
2016-05-18 02:07:18,861 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537238657_0001_1_00_000000_0] started. Is using containerId: [container_1463537238657_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:18,861 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238657_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D82F27793E91497D96F6A5726D2934EA, taskAttemptId=attempt_1463537238657_0001_1_00_000000_0, startTime=1463537238861, containerId=container_1463537238657_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:18,862 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537238657_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537238657_0001_1_00_000000_0
2016-05-18 02:07:18 Starting to run new task attempt: attempt_1463537238657_0001_1_00_000000_0
2016-05-18 02:07:18,862 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537238657_0001_1_00_000000_0
2016-05-18 02:07:18,862 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:18,863 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:18,863 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [918E320DC0424733A237447706EFAC31/CAE15CA3E02A4DE48FAC6625C2C0AFEF] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL, VertexName: D82F27793E91497D96F6A5726D2934EA, VertexParallelism: 1, TaskAttemptID:attempt_1463537238657_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DC879B622DF64C2592E6A0F3E9F814AF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A50FB30328C6408D8DEE0C1E4E9449AD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:18,863 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:18,863 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537238657_0001_1_00_000000_0
2016-05-18 02:07:18,866 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537238657_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:18,867 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537238657_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:18,869 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:18,871 INFO input.MRInput (MRInput.java:initialize(440)) - DC879B622DF64C2592E6A0F3E9F814AF using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:18,871 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: DC879B622DF64C2592E6A0F3E9F814AF
2016-05-18 02:07:18,872 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537238657_0001, with dag id: dag_1463537238657_0001_1
2016-05-18 02:07:18,880 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D82F27793E91497D96F6A5726D2934EA, ordinal: 0
2016-05-18 02:07:18,880 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D82F27793E91497D96F6A5726D2934EA, mem on start (mb), free: 308, total: 386, max: 672
2016-05-18 02:07:18,881 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:18,881 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:18,881 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:18,885 INFO output.MROutput (MROutput.java:initialize(421)) - A50FB30328C6408D8DEE0C1E4E9449AD: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:18,885 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:18,886 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[A50FB30328C6408D8DEE0C1E4E9449AD:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [DC879B622DF64C2592E6A0F3E9F814AF:INPUT:0:org.apache.tez.mapreduce.input.MRInput]
2016-05-18 02:07:18,886 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:18,886 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:18,887 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:18,887 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[A50FB30328C6408D8DEE0C1E4E9449AD:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [DC879B622DF64C2592E6A0F3E9F814AF:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0]
2016-05-18 02:07:18,887 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:18,887 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: DC879B622DF64C2592E6A0F3E9F814AF
2016-05-18 02:07:18,888 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: DC879B622DF64C2592E6A0F3E9F814AF being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:18,888 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:18,888 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:18,888 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:18,889 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537238657_0001_1_00_000000_0
2016-05-18 02:07:18,890 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:18,891 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - DC879B622DF64C2592E6A0F3E9F814AF initialized RecordReader from event
2016-05-18 02:07:18,893 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:18,893 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: DC879B622DF64C2592E6A0F3E9F814AF
2016-05-18 02:07:18,894 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_ALL"], id: A50FB30328C6408D8DEE0C1E4E9449AD
2016-05-18 02:07:18,894 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_ALL"]
2016-05-18 02:07:18,894 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:18,895 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D82F27793E91497D96F6A5726D2934EA, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:18,897 INFO output.MROutput (MROutput.java:close(526)) - A50FB30328C6408D8DEE0C1E4E9449AD closed
2016-05-18 02:07:18,898 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537238657_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:18,899 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537238657_0001_1_00_000000_0
2016-05-18 02:07:18,899 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D82F27793E91497D96F6A5726D2934EA, mem on close (mb), free: 306, total: 386, max: 672
2016-05-18 02:07:18,900 INFO output.MROutput (MROutput.java:close(526)) - A50FB30328C6408D8DEE0C1E4E9449AD closed
2016-05-18 02:07:18,900 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:18 Completed running task attempt: attempt_1463537238657_0001_1_00_000000_0
2016-05-18 02:07:18,901 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537238657_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:18,901 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537238657_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:18,902 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537238657_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537238895, Process_Duration=4, Process_End_Time=1463537238899, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:18,903 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537238657_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:18,903 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238657_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D82F27793E91497D96F6A5726D2934EA, taskAttemptId=attempt_1463537238657_0001_1_00_000000_0, creationTime=1463537238856, allocationTime=1463537238859, startTime=1463537238861, finishTime=1463537238903, timeTaken=42, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:18,905 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537238657_0001_1_00_000000_0
2016-05-18 02:07:18,904 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:18,905 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:18,906 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:18,906 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238657_0001_1][Event:TASK_FINISHED]: vertexName=D82F27793E91497D96F6A5726D2934EA, taskId=task_1463537238657_0001_1_00_000000, startTime=1463537238861, finishTime=1463537238905, timeTaken=44, status=SUCCEEDED, successfulAttemptID=attempt_1463537238657_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537238895, Process_Duration=4, Process_End_Time=1463537238899, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:18,906 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:18,906 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA]
2016-05-18 02:07:18,907 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537238657_0001_00_000001
2016-05-18 02:07:18,907 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238657_0001_1][Event:VERTEX_FINISHED]: vertexName=D82F27793E91497D96F6A5726D2934EA, vertexId=vertex_1463537238657_0001_1_00, initRequestedTime=1463537238841, initedTime=1463537238851, startRequestedTime=1463537238843, startedTime=1463537238854, finishTime=1463537238906, timeTaken=52, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537238895, Process_Duration=4, Process_End_Time=1463537238899, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537238861, firstTasksToStart=[ task_1463537238657_0001_1_00_000000 ], lastTaskFinishTime=1463537238903, lastTasksToFinish=[ task_1463537238657_0001_1_00_000000 ], minTaskDuration=42, maxTaskDuration=42, avgTaskDuration=42.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537238657_0001_1_00_000000 ], longestDurationTasks=[ task_1463537238657_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:18,907 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:18,908 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238657_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537238657_0001_00_000001, stoppedTime=1463537238908, exitStatus=0
2016-05-18 02:07:18,908 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:18,908 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:18,908 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537238657_0001_1
2016-05-18 02:07:18,908 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238657_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537238657_0001_1
2016-05-18 02:07:18,908 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: A50FB30328C6408D8DEE0C1E4E9449AD for vertex: vertex_1463537238657_0001_1_00 [D82F27793E91497D96F6A5726D2934EA], outputName: A50FB30328C6408D8DEE0C1E4E9449AD
2016-05-18 02:07:18,908 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537238657_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:18,909 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537238657_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:18,917 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537238657_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:18,918 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537238657_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:18,918 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537238657_0001_00_000001, Exiting
2016-05-18 02:07:18,918 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537238657_0001_00_000001
2016-05-18 02:07:18,918 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537238657_0001_00_000001
2016-05-18 02:07:18,918 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:A50FB30328C6408D8DEE0C1E4E9449AD of vertex/vertexGroup:D82F27793E91497D96F6A5726D2934EA isVertexGroupOutput:false
2016-05-18 02:07:18,919 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:18,920 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537238657_0001_1, queueSize=0
2016-05-18 02:07:18,920 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537238657_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537238657_0001_1, startTime=1463537238840, finishTime=1463537238919, timeTaken=79, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=160, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537238895, Process_Duration=4, Process_End_Time=1463537238899, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:18,921 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537238657_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:18,921 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537238657_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:18 Completed Dag: dag_1463537238657_0001_1
2016-05-18 02:07:18,921 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537238657_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:18,921 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537238657_0001_1_post
2016-05-18 02:07:18,921 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:18,922 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:18,922 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:18,922 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[918E320DC0424733A237447706EFAC31/CAE15CA3E02A4DE48FAC6625C2C0AFEF] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL, with id=dag_1463537238657_0001_1
2016-05-18 02:07:18,922 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[918E320DC0424733A237447706EFAC31/CAE15CA3E02A4DE48FAC6625C2C0AFEF] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL, with id=dag_1463537238657_0001_1
2016-05-18 02:07:18,922 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537238657_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:18,923 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537238657_0001_00_000001 completed successfully
2016-05-18 02:07:18,934 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL, applicationId=application_1463537238657_0001
2016-05-18 02:07:18,936 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:18,936 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:18,936 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:18,936 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:18,936 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:18,937 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:18,937 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:18,937 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:18,938 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:18,938 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:18,938 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:18,939 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:18,939 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:18,940 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:18,943 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/918E320DC0424733A237447706EFAC31/CAE15CA3E02A4DE48FAC6625C2C0AFEF/.tez/application_1463537238657_0001
2016-05-18 02:07:18,943 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_ALL/_temporary
2016-05-18 02:07:18,944 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.311
2016-05-18 02:07:19,022 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_UNKNOWN_ALL
2016-05-18 02:07:19,031 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 02:07:19,031 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 02:07:19,050 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:19,050 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:19,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-18 02:07:19,101 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-05-18 02:07:19,102 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:19,102 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:19,103 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:19,122 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:19,123 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_-1
2016-05-18 02:07:19,136 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:19,136 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:19,136 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_-1"]
2016-05-18 02:07:19,136 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:19,137 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:19,137 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:19,137 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-18 02:07:19,138 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:19,153 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:19,157 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:19,157 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:19,158 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:19,166 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A89678BDED8D464EA29039F9D3163F71/E00E1FB12E99464EA6249878CA410E1A/.tez/application_1463537239157_0001 doesn't exist and is created
2016-05-18 02:07:19,187 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:19,188 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:19,188 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A89678BDED8D464EA29039F9D3163F71/E00E1FB12E99464EA6249878CA410E1A/.tez/application_1463537239157_0001_wd
2016-05-18 02:07:19,204 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537239157_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:19,205 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:19,205 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:19,206 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:19,206 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:19,206 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:19,206 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:19,207 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:19,209 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:19,209 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:19,209 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:19,209 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:19,209 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:19,209 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:19,210 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:19,210 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:19,210 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:19,210 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:19,210 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:19,211 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:19,212 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:19,213 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:19,213 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:19,213 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A89678BDED8D464EA29039F9D3163F71/E00E1FB12E99464EA6249878CA410E1A/.tez/application_1463537239157_0001_wd/localmode-log-dir
2016-05-18 02:07:19,214 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A89678BDED8D464EA29039F9D3163F71/E00E1FB12E99464EA6249878CA410E1A/.tez/application_1463537239157_0001_wd/localmode-log-dir/history.txt.appattempt_1463537239157_0001_000000, maxErrors=10
2016-05-18 02:07:19,214 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A89678BDED8D464EA29039F9D3163F71/E00E1FB12E99464EA6249878CA410E1A/.tez/application_1463537239157_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:19,214 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537239157_0001_000000, appSubmitTime=1463537239204, launchTime=1463537239204
2016-05-18 02:07:19,222 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:19,225 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:45565
2016-05-18 02:07:19,225 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537239157
2016-05-18 02:07:19,227 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537239157_0001_000000, startTime=1463537239227
2016-05-18 02:07:19,227 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:19,288 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:19,288 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:19,288 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.-1_VALUES_-1, applicationId=application_1463537239157_0001, dagName=[A89678BDED8D464EA29039F9D3163F71/E00E1FB12E99464EA6249878CA410E1A] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-18 02:07:19,310 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [A89678BDED8D464EA29039F9D3163F71/E00E1FB12E99464EA6249878CA410E1A] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-18 02:07:19,310 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:19,312 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537239157_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A89678BDED8D464EA29039F9D3163F71/E00E1FB12E99464EA6249878CA410E1A/.tez/application_1463537239157_0001_wd/localmode-log-dir/dag_1463537239157_0001_1.dot
2016-05-18 02:07:19,313 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A89678BDED8D464EA29039F9D3163F71/E00E1FB12E99464EA6249878CA410E1A/.tez/application_1463537239157_0001_wd/localmode-log-dir/dag_1463537239157_0001_1-tez-dag.pb.txt
2016-05-18 02:07:19,317 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537239157_0001_1
2016-05-18 02:07:19,318 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:19,318 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [A89678BDED8D464EA29039F9D3163F71/E00E1FB12E99464EA6249878CA410E1A] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-18 02:07:19 Running Dag: dag_1463537239157_0001_1
2016-05-18 02:07:19,334 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239157_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537239157_0001_1, submitTime=1463537239310
2016-05-18 02:07:19,335 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A89678BDED8D464EA29039F9D3163F71/E00E1FB12E99464EA6249878CA410E1A/.tez/application_1463537239157_0001/tez-conf.pb" } size: 40947 timestamp: 1463537239000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:19,335 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B]
2016-05-18 02:07:19,336 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B]
2016-05-18 02:07:19,336 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:19,336 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:19,337 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239157_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537239157_0001_1, initTime=1463537239336
2016-05-18 02:07:19,337 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537239157_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:19,337 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:19,338 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239157_0001_1][Event:DAG_STARTED]: dagID=dag_1463537239157_0001_1, startTime=1463537239338
2016-05-18 02:07:19,338 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537239157_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:19,338 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: AEACC16A22A44EC39B8B0809A4525C2B : {F8B48314B260439B8102A559B14E402D={InputName=F8B48314B260439B8102A559B14E402D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:19,338 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.-1_VALUES_-1, applicationId=application_1463537239157_0001, dagName=[A89678BDED8D464EA29039F9D3163F71/E00E1FB12E99464EA6249878CA410E1A] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-18 02:07:19,338 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: F8B48314B260439B8102A559B14E402D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:19,339 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B]
2016-05-18 02:07:19,339 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B]
2016-05-18 02:07:19,339 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B]
2016-05-18 02:07:19,339 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B]
2016-05-18 02:07:19,340 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:19,341 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: F8B48314B260439B8102A559B14E402D on vertex vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B]
2016-05-18 02:07:19,343 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input F8B48314B260439B8102A559B14E402D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:19,346 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:19,346 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:19,347 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:19,347 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:19,347 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: F8B48314B260439B8102A559B14E402D on vertex vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B]
2016-05-18 02:07:19,349 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:19,349 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {F8B48314B260439B8102A559B14E402D=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:19,350 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B] parallelism set to 1
2016-05-18 02:07:19,350 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:19,350 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B], numAdditionalOutputs=1
2016-05-18 02:07:19,351 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=AF39853349344179B2EC7D9695472F23, vertex=vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:19,353 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for AEACC16A22A44EC39B8B0809A4525C2B:AF39853349344179B2EC7D9695472F23 using oldmapred API
2016-05-18 02:07:19,353 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=AF39853349344179B2EC7D9695472F23, vertexName=AEACC16A22A44EC39B8B0809A4525C2B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:19,354 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239157_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AEACC16A22A44EC39B8B0809A4525C2B, vertexId=vertex_1463537239157_0001_1_00, initRequestedTime=1463537239338, initedTime=1463537239350, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:19,355 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B] with distanceFromRoot: 0
2016-05-18 02:07:19,355 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:19,355 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239157_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537239157_0001_1_00, startRequestedTime=1463537239340, startedTime=1463537239355
2016-05-18 02:07:19,356 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239157_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537239157_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:19,356 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:19,357 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in AEACC16A22A44EC39B8B0809A4525C2B
2016-05-18 02:07:19,357 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B]
2016-05-18 02:07:19,358 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239157_0001_1][Event:TASK_STARTED]: vertexName=AEACC16A22A44EC39B8B0809A4525C2B, taskId=task_1463537239157_0001_1_00_000000, scheduledTime=1463537239357, launchTime=1463537239357
2016-05-18 02:07:19,358 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B]
2016-05-18 02:07:19,359 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:19,361 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537239157_0001_00_000001, launchTime=1463537239361
2016-05-18 02:07:19,362 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537239157_0001_00_000001
2016-05-18 02:07:19,363 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537239157_0001_1_00_000000_0] started. Is using containerId: [container_1463537239157_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:19,364 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239157_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AEACC16A22A44EC39B8B0809A4525C2B, taskAttemptId=attempt_1463537239157_0001_1_00_000000_0, startTime=1463537239363, containerId=container_1463537239157_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:19,365 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537239157_0001, with dag id: dag_1463537239157_0001_1
2016-05-18 02:07:19,365 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537239157_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537239157_0001_1_00_000000_0
2016-05-18 02:07:19 Starting to run new task attempt: attempt_1463537239157_0001_1_00_000000_0
2016-05-18 02:07:19,366 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537239157_0001_1_00_000000_0
2016-05-18 02:07:19,367 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:19,368 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:19,369 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A89678BDED8D464EA29039F9D3163F71/E00E1FB12E99464EA6249878CA410E1A] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1, VertexName: AEACC16A22A44EC39B8B0809A4525C2B, VertexParallelism: 1, TaskAttemptID:attempt_1463537239157_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F8B48314B260439B8102A559B14E402D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AF39853349344179B2EC7D9695472F23, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:19,369 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:19,370 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537239157_0001_1_00_000000_0
2016-05-18 02:07:19,372 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537239157_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:19,372 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537239157_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:19,375 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:19,380 INFO input.MRInput (MRInput.java:initialize(440)) - F8B48314B260439B8102A559B14E402D using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:19,380 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: F8B48314B260439B8102A559B14E402D
2016-05-18 02:07:19,387 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AEACC16A22A44EC39B8B0809A4525C2B, ordinal: 0
2016-05-18 02:07:19,387 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AEACC16A22A44EC39B8B0809A4525C2B, mem on start (mb), free: 215, total: 386, max: 672
2016-05-18 02:07:19,387 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:19,387 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:19,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:19,393 INFO output.MROutput (MROutput.java:initialize(421)) - AF39853349344179B2EC7D9695472F23: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:19,394 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:19,394 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[F8B48314B260439B8102A559B14E402D:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [AF39853349344179B2EC7D9695472F23:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:19,395 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:19,395 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:19,395 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:19,395 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[F8B48314B260439B8102A559B14E402D:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [AF39853349344179B2EC7D9695472F23:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:19,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:19,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: F8B48314B260439B8102A559B14E402D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:19,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:19,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:19,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: F8B48314B260439B8102A559B14E402D
2016-05-18 02:07:19,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:19,396 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537239157_0001_1_00_000000_0
2016-05-18 02:07:19,398 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:19,399 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - F8B48314B260439B8102A559B14E402D initialized RecordReader from event
2016-05-18 02:07:19,401 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:19,401 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: F8B48314B260439B8102A559B14E402D
2016-05-18 02:07:19,402 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_-1"], id: AF39853349344179B2EC7D9695472F23
2016-05-18 02:07:19,402 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_-1"]
2016-05-18 02:07:19,402 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:19,402 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AEACC16A22A44EC39B8B0809A4525C2B, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:19,404 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 02:07:19,406 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 02:07:19,407 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 02:07:19,408 INFO output.MROutput (MROutput.java:close(526)) - AF39853349344179B2EC7D9695472F23 closed
2016-05-18 02:07:19,408 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537239157_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:19,409 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537239157_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:19,409 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537239157_0001_1_00_000000_0: Counters: 8 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463537239403, Process_Duration=6, Process_End_Time=1463537239409, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-18 02:07:19,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(817)) - Joining on EventRouter
2016-05-18 02:07:19,410 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AEACC16A22A44EC39B8B0809A4525C2B, mem on close (mb), free: 212, total: 386, max: 672
2016-05-18 02:07:19,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(851)) - Closed processor for vertex=AEACC16A22A44EC39B8B0809A4525C2B, index=0, interruptedStatus=false
2016-05-18 02:07:19,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(887)) - Closed input for vertex=AEACC16A22A44EC39B8B0809A4525C2B, sourceVertex=F8B48314B260439B8102A559B14E402D, interruptedStatus=true
2016-05-18 02:07:19,410 INFO output.MROutput (MROutput.java:close(526)) - AF39853349344179B2EC7D9695472F23 closed
2016-05-18 02:07:19,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(911)) - Closed input for vertex=AEACC16A22A44EC39B8B0809A4525C2B, sourceVertex=AF39853349344179B2EC7D9695472F23, interruptedStatus=true
2016-05-18 02:07:19,418 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537239157_0001_1_00_000000_0 : TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 02:07:19,418 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 02:07:19,419 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537239157_0001_00_000001
2016-05-18 02:07:19,419 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537239157_0001_00_000001
2016-05-18 02:07:19,419 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537239157_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, errorMessage='TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation'}
2016-05-18 02:07:19,420 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(294)) - Container: container_1463537239157_0001_00_000001 completed but with errors
2016-05-18 02:07:19,420 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239157_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AEACC16A22A44EC39B8B0809A4525C2B, taskAttemptId=attempt_1463537239157_0001_1_00_000000_0, creationTime=1463537239358, allocationTime=1463537239360, startTime=1463537239363, finishTime=1463537239418, timeTaken=55, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task: attempt_1463537239157_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
, counters=Counters: 9, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463537239403, Process_Duration=6, Process_End_Time=1463537239409, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-18 02:07:19,422 INFO container.AMContainerImpl (AMContainerImpl.java:transition(704)) - Container container_1463537239157_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
2016-05-18 02:07:19,423 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239157_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537239157_0001_00_000001, stoppedTime=1463537239422, exitStatus=1
2016-05-18 02:07:19,423 INFO impl.TaskImpl (TaskImpl.java:transition(1170)) - Failing task: task_1463537239157_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-18 02:07:19,424 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239157_0001_1][Event:TASK_FINISHED]: vertexName=AEACC16A22A44EC39B8B0809A4525C2B, taskId=task_1463537239157_0001_1_00_000000, startTime=1463537239363, finishTime=1463537239423, timeTaken=60, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463537239157_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
], counters=Counters: 0
2016-05-18 02:07:19,427 INFO impl.VertexImpl (VertexImpl.java:transition(3353)) - Failing vertex: vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B] because task failed: task_1463537239157_0001_1_00_000000
2016-05-18 02:07:19,428 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(2102)) - Killing tasks in vertex: vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B] due to trigger: OWN_TASK_FAILURE
2016-05-18 02:07:19,428 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B], tasks=1, failed=1, killed=0, success=0, completed=1, commits=0, err=OWN_TASK_FAILURE
2016-05-18 02:07:19,429 INFO impl.VertexImpl (VertexImpl.java:finishWithTerminationCause(2084)) - Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
2016-05-18 02:07:19,429 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3087)) - Invoking committer abort for vertex, vertexId=vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B]
2016-05-18 02:07:19,430 INFO impl.VertexImpl (VertexImpl.java:run(3095)) - Invoking committer abort for output=AF39853349344179B2EC7D9695472F23, vertexId=vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B]
2016-05-18 02:07:19,434 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239157_0001_1][Event:VERTEX_FINISHED]: vertexName=AEACC16A22A44EC39B8B0809A4525C2B, vertexId=vertex_1463537239157_0001_1_00, initRequestedTime=1463537239338, initedTime=1463537239350, startRequestedTime=1463537239340, startedTime=1463537239355, finishTime=1463537239433, timeTaken=78, status=FAILED, diagnostics=Task failed, taskId=task_1463537239157_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463537239157_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]]
Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
Vertex vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B] killed/failed due to:OWN_TASK_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=1463537239363, firstTasksToStart=[ task_1463537239157_0001_1_00_000000 ], lastTaskFinishTime=1463537239418, lastTasksToFinish=[ task_1463537239157_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-18 02:07:19,434 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-18 02:07:19,438 INFO node.AMNodeImpl (AMNodeImpl.java:transition(298)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463537239157_0001_1_00_000000_0 failed: true container: container_1463537239157_0001_00_000001 numFailedTAs: 0
2016-05-18 02:07:19,440 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-18 02:07:19,440 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=VERTEX_FAILURE
2016-05-18 02:07:19,440 INFO impl.DAGImpl (DAGImpl.java:finishWithTerminationCause(1350)) - DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
2016-05-18 02:07:19,441 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537239157_0001_1, queueSize=0
2016-05-18 02:07:19,443 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239157_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537239157_0001_1, startTime=1463537239337, finishTime=1463537239441, timeTaken=104, status=FAILED, diagnostics=Vertex failed, vertexName=AEACC16A22A44EC39B8B0809A4525C2B, vertexId=vertex_1463537239157_0001_1_00, diagnostics=[Task failed, taskId=task_1463537239157_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463537239157_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B] killed/failed due to:OWN_TASK_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0, counters=Counters: 4, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, AM_CPU_MILLISECONDS=210, AM_GC_TIME_MILLIS=0
2016-05-18 02:07:19,444 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3082)) - Ignoring multiple aborts for vertex: vertex_1463537239157_0001_1_00 [AEACC16A22A44EC39B8B0809A4525C2B]
2016-05-18 02:07:19,444 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537239157_0001_1 finished with state: FAILED
2016-05-18 02:07:19,445 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537239157_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:19 Completed Dag: dag_1463537239157_0001_1
2016-05-18 02:07:19,446 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537239157_0001_1, dagState=FAILED
2016-05-18 02:07:19,446 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537239157_0001_1_post
2016-05-18 02:07:19,446 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:19,447 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:19,447 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:19,448 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[A89678BDED8D464EA29039F9D3163F71/E00E1FB12E99464EA6249878CA410E1A] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1, with id=dag_1463537239157_0001_1
2016-05-18 02:07:19,451 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[A89678BDED8D464EA29039F9D3163F71/E00E1FB12E99464EA6249878CA410E1A] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1, with id=dag_1463537239157_0001_1
2016-05-18 02:07:19,454 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...eld'_RESULTS.-1_VALUES_-1, applicationId=application_1463537239157_0001
2016-05-18 02:07:19,457 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:19,458 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:19,458 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:19,458 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:19,458 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:19,459 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:19,459 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:19,459 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:19,459 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:19,460 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:19,460 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:19,460 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:19,461 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:19,461 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:19,464 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A89678BDED8D464EA29039F9D3163F71/E00E1FB12E99464EA6249878CA410E1A/.tez/application_1463537239157_0001
2016-05-18 02:07:19,464 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-18 02:07:19,465 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-18 02:07:19,467 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-18 02:07:19,467 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_-1/_temporary
2016-05-18 02:07:19,468 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.331
2016-05-18 02:07:19,469 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_VALUES_-1
2016-05-18 02:07:19,497 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:19,497 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:19,538 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.008
2016-05-18 02:07:19,540 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.010
2016-05-18 02:07:19,541 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 02:07:19,541 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 02:07:19,542 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_VALUES_'field2'
2016-05-18 02:07:19,586 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:19,586 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:19,627 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-18 02:07:19,643 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-05-18 02:07:19,644 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:19,644 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:19,645 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:19,675 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:19,676 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_RESULTS
2016-05-18 02:07:19,689 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:19,689 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:19,689 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_RESULTS"]
2016-05-18 02:07:19,690 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:19,690 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:19,690 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:19,691 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-18 02:07:19,691 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:19,706 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:19,710 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:19,711 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:19,711 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:19,720 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B5496768C6024AF894DAED1B3EB0BC17/FAEC224B9475410DA6B5C228447FF09B/.tez/application_1463537239711_0001 doesn't exist and is created
2016-05-18 02:07:19,740 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:19,740 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:19,741 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B5496768C6024AF894DAED1B3EB0BC17/FAEC224B9475410DA6B5C228447FF09B/.tez/application_1463537239711_0001_wd
2016-05-18 02:07:19,758 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537239711_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:19,758 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:19,758 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:19,760 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:19,761 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:19,761 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:19,761 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:19,762 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:19,764 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:19,764 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:19,764 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:19,764 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:19,764 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:19,765 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:19,765 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:19,765 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:19,766 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:19,766 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:19,766 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:19,766 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:19,768 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:19,769 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:19,769 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:19,769 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B5496768C6024AF894DAED1B3EB0BC17/FAEC224B9475410DA6B5C228447FF09B/.tez/application_1463537239711_0001_wd/localmode-log-dir
2016-05-18 02:07:19,769 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B5496768C6024AF894DAED1B3EB0BC17/FAEC224B9475410DA6B5C228447FF09B/.tez/application_1463537239711_0001_wd/localmode-log-dir/history.txt.appattempt_1463537239711_0001_000000, maxErrors=10
2016-05-18 02:07:19,770 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B5496768C6024AF894DAED1B3EB0BC17/FAEC224B9475410DA6B5C228447FF09B/.tez/application_1463537239711_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:19,770 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537239711_0001_000000, appSubmitTime=1463537239757, launchTime=1463537239757
2016-05-18 02:07:19,774 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:19,779 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:53748
2016-05-18 02:07:19,796 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537239711
2016-05-18 02:07:19,799 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537239711_0001_000000, startTime=1463537239799
2016-05-18 02:07:19,799 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:19,816 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 02:07:19,816 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 02:07:19,840 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:19,840 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:19,841 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...RESULTS.-1_VALUES_RESULTS, applicationId=application_1463537239711_0001, dagName=[B5496768C6024AF894DAED1B3EB0BC17/FAEC224B9475410DA6B5C228447FF09B] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-18 02:07:19,865 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [B5496768C6024AF894DAED1B3EB0BC17/FAEC224B9475410DA6B5C228447FF09B] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-18 02:07:19,865 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:19,866 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537239711_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B5496768C6024AF894DAED1B3EB0BC17/FAEC224B9475410DA6B5C228447FF09B/.tez/application_1463537239711_0001_wd/localmode-log-dir/dag_1463537239711_0001_1.dot
2016-05-18 02:07:19,867 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B5496768C6024AF894DAED1B3EB0BC17/FAEC224B9475410DA6B5C228447FF09B/.tez/application_1463537239711_0001_wd/localmode-log-dir/dag_1463537239711_0001_1-tez-dag.pb.txt
2016-05-18 02:07:19,871 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537239711_0001_1
2016-05-18 02:07:19,871 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:19,871 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [B5496768C6024AF894DAED1B3EB0BC17/FAEC224B9475410DA6B5C228447FF09B] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-18 02:07:19 Running Dag: dag_1463537239711_0001_1
2016-05-18 02:07:19,887 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239711_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537239711_0001_1, submitTime=1463537239865
2016-05-18 02:07:19,888 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B5496768C6024AF894DAED1B3EB0BC17/FAEC224B9475410DA6B5C228447FF09B/.tez/application_1463537239711_0001/tez-conf.pb" } size: 40947 timestamp: 1463537239000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:19,888 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA]
2016-05-18 02:07:19,889 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA]
2016-05-18 02:07:19,889 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:19,889 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:19,889 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239711_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537239711_0001_1, initTime=1463537239889
2016-05-18 02:07:19,891 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537239711_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:19,891 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:19,891 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239711_0001_1][Event:DAG_STARTED]: dagID=dag_1463537239711_0001_1, startTime=1463537239891
2016-05-18 02:07:19,891 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...RESULTS.-1_VALUES_RESULTS, applicationId=application_1463537239711_0001, dagName=[B5496768C6024AF894DAED1B3EB0BC17/FAEC224B9475410DA6B5C228447FF09B] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-18 02:07:19,891 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537239711_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:19,892 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: D0BD7397FF604564A75B06C059DEABAA : {8BC4011A90DA453CA39C428706B35B07={InputName=8BC4011A90DA453CA39C428706B35B07}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:19,892 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 8BC4011A90DA453CA39C428706B35B07, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:19,892 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA]
2016-05-18 02:07:19,892 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA]
2016-05-18 02:07:19,892 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA]
2016-05-18 02:07:19,892 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA]
2016-05-18 02:07:19,893 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:19,893 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 8BC4011A90DA453CA39C428706B35B07 on vertex vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA]
2016-05-18 02:07:19,895 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 8BC4011A90DA453CA39C428706B35B07 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:19,898 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:19,898 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:19,898 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:19,898 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:19,899 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 8BC4011A90DA453CA39C428706B35B07 on vertex vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA]
2016-05-18 02:07:19,900 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:19,900 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {8BC4011A90DA453CA39C428706B35B07=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:19,901 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA] parallelism set to 1
2016-05-18 02:07:19,901 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:19,901 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA], numAdditionalOutputs=1
2016-05-18 02:07:19,902 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=1BEBE54A1F984062BCF9260A344C426C, vertex=vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:19,904 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D0BD7397FF604564A75B06C059DEABAA:1BEBE54A1F984062BCF9260A344C426C using oldmapred API
2016-05-18 02:07:19,904 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=1BEBE54A1F984062BCF9260A344C426C, vertexName=D0BD7397FF604564A75B06C059DEABAA, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:19,905 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239711_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D0BD7397FF604564A75B06C059DEABAA, vertexId=vertex_1463537239711_0001_1_00, initRequestedTime=1463537239892, initedTime=1463537239901, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:19,905 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA] with distanceFromRoot: 0
2016-05-18 02:07:19,905 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:19,906 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239711_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537239711_0001_1_00, startRequestedTime=1463537239893, startedTime=1463537239906
2016-05-18 02:07:19,906 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239711_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537239711_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:19,907 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:19,907 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in D0BD7397FF604564A75B06C059DEABAA
2016-05-18 02:07:19,907 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA]
2016-05-18 02:07:19,908 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239711_0001_1][Event:TASK_STARTED]: vertexName=D0BD7397FF604564A75B06C059DEABAA, taskId=task_1463537239711_0001_1_00_000000, scheduledTime=1463537239907, launchTime=1463537239907
2016-05-18 02:07:19,908 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA]
2016-05-18 02:07:19,909 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:19,910 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537239711_0001_00_000001, launchTime=1463537239910
2016-05-18 02:07:19,911 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537239711_0001_00_000001
2016-05-18 02:07:19,912 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537239711_0001_1_00_000000_0] started. Is using containerId: [container_1463537239711_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:19,912 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239711_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D0BD7397FF604564A75B06C059DEABAA, taskAttemptId=attempt_1463537239711_0001_1_00_000000_0, startTime=1463537239912, containerId=container_1463537239711_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:19,913 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537239711_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537239711_0001_1_00_000000_0
2016-05-18 02:07:19 Starting to run new task attempt: attempt_1463537239711_0001_1_00_000000_0
2016-05-18 02:07:19,914 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537239711_0001_1_00_000000_0
2016-05-18 02:07:19,914 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:19,914 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:19,915 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B5496768C6024AF894DAED1B3EB0BC17/FAEC224B9475410DA6B5C228447FF09B] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS, VertexName: D0BD7397FF604564A75B06C059DEABAA, VertexParallelism: 1, TaskAttemptID:attempt_1463537239711_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8BC4011A90DA453CA39C428706B35B07, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=1BEBE54A1F984062BCF9260A344C426C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:19,915 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537239711_0001, with dag id: dag_1463537239711_0001_1
2016-05-18 02:07:19,915 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:19,916 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537239711_0001_1_00_000000_0
2016-05-18 02:07:19,917 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537239711_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:19,917 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537239711_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:19,925 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:19,932 INFO input.MRInput (MRInput.java:initialize(440)) - 8BC4011A90DA453CA39C428706B35B07 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:19,932 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 8BC4011A90DA453CA39C428706B35B07
2016-05-18 02:07:19,935 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D0BD7397FF604564A75B06C059DEABAA, ordinal: 0
2016-05-18 02:07:19,935 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D0BD7397FF604564A75B06C059DEABAA, mem on start (mb), free: 322, total: 385, max: 672
2016-05-18 02:07:19,935 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:19,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:19,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:19,944 INFO output.MROutput (MROutput.java:initialize(421)) - 1BEBE54A1F984062BCF9260A344C426C: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:19,944 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:19,944 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[8BC4011A90DA453CA39C428706B35B07:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [1BEBE54A1F984062BCF9260A344C426C:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:19,945 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:19,945 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:19,945 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:19,945 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[8BC4011A90DA453CA39C428706B35B07:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [1BEBE54A1F984062BCF9260A344C426C:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:19,945 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:19,945 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 8BC4011A90DA453CA39C428706B35B07 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:19,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 8BC4011A90DA453CA39C428706B35B07
2016-05-18 02:07:19,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:19,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:19,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:19,946 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537239711_0001_1_00_000000_0
2016-05-18 02:07:19,948 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:19,949 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 8BC4011A90DA453CA39C428706B35B07 initialized RecordReader from event
2016-05-18 02:07:19,949 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:19,950 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 8BC4011A90DA453CA39C428706B35B07
2016-05-18 02:07:19,950 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_RESULTS"], id: 1BEBE54A1F984062BCF9260A344C426C
2016-05-18 02:07:19,950 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_RESULTS"]
2016-05-18 02:07:19,951 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:19,951 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D0BD7397FF604564A75B06C059DEABAA, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:19,951 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 02:07:19,953 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 02:07:19,955 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 02:07:19,956 INFO output.MROutput (MROutput.java:close(526)) - 1BEBE54A1F984062BCF9260A344C426C closed
2016-05-18 02:07:19,957 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537239711_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:19,958 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537239711_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:19,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537239711_0001_1_00_000000_0: Counters: 8 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463537239951, Process_Duration=6, Process_End_Time=1463537239957, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-18 02:07:19,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(817)) - Joining on EventRouter
2016-05-18 02:07:19,958 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D0BD7397FF604564A75B06C059DEABAA, mem on close (mb), free: 320, total: 385, max: 672
2016-05-18 02:07:19,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(851)) - Closed processor for vertex=D0BD7397FF604564A75B06C059DEABAA, index=0, interruptedStatus=false
2016-05-18 02:07:19,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(887)) - Closed input for vertex=D0BD7397FF604564A75B06C059DEABAA, sourceVertex=8BC4011A90DA453CA39C428706B35B07, interruptedStatus=true
2016-05-18 02:07:19,959 INFO output.MROutput (MROutput.java:close(526)) - 1BEBE54A1F984062BCF9260A344C426C closed
2016-05-18 02:07:19,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(911)) - Closed input for vertex=D0BD7397FF604564A75B06C059DEABAA, sourceVertex=1BEBE54A1F984062BCF9260A344C426C, interruptedStatus=true
2016-05-18 02:07:19,960 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537239711_0001_1_00_000000_0 : TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 02:07:19,960 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 02:07:19,960 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537239711_0001_00_000001
2016-05-18 02:07:19,960 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239711_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D0BD7397FF604564A75B06C059DEABAA, taskAttemptId=attempt_1463537239711_0001_1_00_000000_0, creationTime=1463537239908, allocationTime=1463537239910, startTime=1463537239912, finishTime=1463537239960, timeTaken=48, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task: attempt_1463537239711_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
, counters=Counters: 9, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463537239951, Process_Duration=6, Process_End_Time=1463537239957, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-18 02:07:19,961 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537239711_0001_00_000001
2016-05-18 02:07:19,961 INFO impl.TaskImpl (TaskImpl.java:transition(1170)) - Failing task: task_1463537239711_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-18 02:07:19,961 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239711_0001_1][Event:TASK_FINISHED]: vertexName=D0BD7397FF604564A75B06C059DEABAA, taskId=task_1463537239711_0001_1_00_000000, startTime=1463537239912, finishTime=1463537239961, timeTaken=49, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463537239711_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
], counters=Counters: 0
2016-05-18 02:07:19,964 INFO impl.VertexImpl (VertexImpl.java:transition(3353)) - Failing vertex: vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA] because task failed: task_1463537239711_0001_1_00_000000
2016-05-18 02:07:19,964 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(2102)) - Killing tasks in vertex: vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA] due to trigger: OWN_TASK_FAILURE
2016-05-18 02:07:19,964 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537239711_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, errorMessage='TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation'}
2016-05-18 02:07:19,964 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA], tasks=1, failed=1, killed=0, success=0, completed=1, commits=0, err=OWN_TASK_FAILURE
2016-05-18 02:07:19,964 INFO impl.VertexImpl (VertexImpl.java:finishWithTerminationCause(2084)) - Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
2016-05-18 02:07:19,964 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(294)) - Container: container_1463537239711_0001_00_000001 completed but with errors
2016-05-18 02:07:19,964 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3087)) - Invoking committer abort for vertex, vertexId=vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA]
2016-05-18 02:07:19,965 INFO impl.VertexImpl (VertexImpl.java:run(3095)) - Invoking committer abort for output=1BEBE54A1F984062BCF9260A344C426C, vertexId=vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA]
2016-05-18 02:07:19,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239711_0001_1][Event:VERTEX_FINISHED]: vertexName=D0BD7397FF604564A75B06C059DEABAA, vertexId=vertex_1463537239711_0001_1_00, initRequestedTime=1463537239892, initedTime=1463537239901, startRequestedTime=1463537239893, startedTime=1463537239906, finishTime=1463537239966, timeTaken=60, status=FAILED, diagnostics=Task failed, taskId=task_1463537239711_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463537239711_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]]
Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
Vertex vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA] killed/failed due to:OWN_TASK_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=1463537239912, firstTasksToStart=[ task_1463537239711_0001_1_00_000000 ], lastTaskFinishTime=1463537239960, lastTasksToFinish=[ task_1463537239711_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-18 02:07:19,967 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-18 02:07:19,968 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239711_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537239711_0001_00_000001, stoppedTime=1463537239968, exitStatus=-1000
2016-05-18 02:07:19,968 INFO node.AMNodeImpl (AMNodeImpl.java:transition(298)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463537239711_0001_1_00_000000_0 failed: true container: container_1463537239711_0001_00_000001 numFailedTAs: 0
2016-05-18 02:07:19,968 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-18 02:07:19,969 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=VERTEX_FAILURE
2016-05-18 02:07:19,969 INFO impl.DAGImpl (DAGImpl.java:finishWithTerminationCause(1350)) - DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
2016-05-18 02:07:19,969 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537239711_0001_1, queueSize=0
2016-05-18 02:07:19,970 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537239711_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537239711_0001_1, startTime=1463537239891, finishTime=1463537239969, timeTaken=78, status=FAILED, diagnostics=Vertex failed, vertexName=D0BD7397FF604564A75B06C059DEABAA, vertexId=vertex_1463537239711_0001_1_00, diagnostics=[Task failed, taskId=task_1463537239711_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463537239711_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA] killed/failed due to:OWN_TASK_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0, counters=Counters: 4, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, AM_CPU_MILLISECONDS=150, AM_GC_TIME_MILLIS=0
2016-05-18 02:07:19,971 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3082)) - Ignoring multiple aborts for vertex: vertex_1463537239711_0001_1_00 [D0BD7397FF604564A75B06C059DEABAA]
2016-05-18 02:07:19,971 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537239711_0001_1 finished with state: FAILED
2016-05-18 02:07:19,971 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537239711_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:19,972 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(260)) - Ignoring stop request for containerId: container_1463537239711_0001_00_000001
2016-05-18 02:07:19,973 INFO container.AMContainerImpl (AMContainerImpl.java:transition(993)) - Container container_1463537239711_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
2016-05-18 02:07:19 Completed Dag: dag_1463537239711_0001_1
2016-05-18 02:07:19,976 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537239711_0001_1, dagState=FAILED
2016-05-18 02:07:19,976 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537239711_0001_1_post
2016-05-18 02:07:19,976 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:19,978 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 2
2016-05-18 02:07:19,978 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:19,978 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[B5496768C6024AF894DAED1B3EB0BC17/FAEC224B9475410DA6B5C228447FF09B] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS, with id=dag_1463537239711_0001_1
2016-05-18 02:07:19,979 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...RESULTS.-1_VALUES_RESULTS, applicationId=application_1463537239711_0001
2016-05-18 02:07:19,981 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[B5496768C6024AF894DAED1B3EB0BC17/FAEC224B9475410DA6B5C228447FF09B] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS, with id=dag_1463537239711_0001_1
2016-05-18 02:07:19,982 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:19,982 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:19,982 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:19,982 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:19,982 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:19,982 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:19,984 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:19,986 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:19,986 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:19,986 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:19,986 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:19,986 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:19,986 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:19,988 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:19,992 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B5496768C6024AF894DAED1B3EB0BC17/FAEC224B9475410DA6B5C228447FF09B/.tez/application_1463537239711_0001
2016-05-18 02:07:19,992 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-18 02:07:19,993 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-18 02:07:19,993 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-18 02:07:19,994 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_RESULTS/_temporary
2016-05-18 02:07:19,994 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.305
2016-05-18 02:07:19,995 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_VALUES_RESULTS
2016-05-18 02:07:20,025 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:20,025 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:20,072 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-18 02:07:20,079 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-05-18 02:07:20,080 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:20,080 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:20,081 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:20,107 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:20,108 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_ALL
2016-05-18 02:07:20,123 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:20,124 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:20,124 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_ALL"]
2016-05-18 02:07:20,124 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:20,125 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:20,125 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:20,126 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-18 02:07:20,126 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:20,141 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:20,145 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:20,145 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:20,146 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:20,154 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/C3505A0A497547CF9534C174CF50AF0B/E66FE1A2BFAD416B9DBE47CD4705FC18/.tez/application_1463537240145_0001 doesn't exist and is created
2016-05-18 02:07:20,174 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:20,175 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:20,176 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/C3505A0A497547CF9534C174CF50AF0B/E66FE1A2BFAD416B9DBE47CD4705FC18/.tez/application_1463537240145_0001_wd
2016-05-18 02:07:20,191 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537240145_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:20,192 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:20,192 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:20,194 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:20,194 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:20,194 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:20,194 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:20,196 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:20,198 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:20,198 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:20,198 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:20,198 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:20,198 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:20,198 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:20,198 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:20,199 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:20,199 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:20,199 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:20,200 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:20,200 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:20,202 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:20,203 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:20,203 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:20,203 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/C3505A0A497547CF9534C174CF50AF0B/E66FE1A2BFAD416B9DBE47CD4705FC18/.tez/application_1463537240145_0001_wd/localmode-log-dir
2016-05-18 02:07:20,204 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/C3505A0A497547CF9534C174CF50AF0B/E66FE1A2BFAD416B9DBE47CD4705FC18/.tez/application_1463537240145_0001_wd/localmode-log-dir/history.txt.appattempt_1463537240145_0001_000000, maxErrors=10
2016-05-18 02:07:20,204 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/C3505A0A497547CF9534C174CF50AF0B/E66FE1A2BFAD416B9DBE47CD4705FC18/.tez/application_1463537240145_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:20,204 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537240145_0001_000000, appSubmitTime=1463537240191, launchTime=1463537240191
2016-05-18 02:07:20,209 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:20,211 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:47472
2016-05-18 02:07:20,213 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537240145
2016-05-18 02:07:20,215 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537240145_0001_000000, startTime=1463537240215
2016-05-18 02:07:20,215 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:20,275 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:20,275 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:20,275 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.-1_VALUES_ALL, applicationId=application_1463537240145_0001, dagName=[C3505A0A497547CF9534C174CF50AF0B/E66FE1A2BFAD416B9DBE47CD4705FC18] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-18 02:07:20,293 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [C3505A0A497547CF9534C174CF50AF0B/E66FE1A2BFAD416B9DBE47CD4705FC18] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-18 02:07:20,293 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:20,294 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537240145_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/C3505A0A497547CF9534C174CF50AF0B/E66FE1A2BFAD416B9DBE47CD4705FC18/.tez/application_1463537240145_0001_wd/localmode-log-dir/dag_1463537240145_0001_1.dot
2016-05-18 02:07:20,295 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/C3505A0A497547CF9534C174CF50AF0B/E66FE1A2BFAD416B9DBE47CD4705FC18/.tez/application_1463537240145_0001_wd/localmode-log-dir/dag_1463537240145_0001_1-tez-dag.pb.txt
2016-05-18 02:07:20,298 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537240145_0001_1
2016-05-18 02:07:20,298 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:20,298 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [C3505A0A497547CF9534C174CF50AF0B/E66FE1A2BFAD416B9DBE47CD4705FC18] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-18 02:07:20 Running Dag: dag_1463537240145_0001_1
2016-05-18 02:07:20,313 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240145_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537240145_0001_1, submitTime=1463537240293
2016-05-18 02:07:20,314 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/C3505A0A497547CF9534C174CF50AF0B/E66FE1A2BFAD416B9DBE47CD4705FC18/.tez/application_1463537240145_0001/tez-conf.pb" } size: 40947 timestamp: 1463537240000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:20,314 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075]
2016-05-18 02:07:20,314 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075]
2016-05-18 02:07:20,314 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:20,314 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:20,315 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240145_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537240145_0001_1, initTime=1463537240315
2016-05-18 02:07:20,315 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537240145_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:20,316 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:20,316 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240145_0001_1][Event:DAG_STARTED]: dagID=dag_1463537240145_0001_1, startTime=1463537240316
2016-05-18 02:07:20,316 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537240145_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:20,316 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: D9231161C767483CB157AB7C4DE55075 : {76EFF2F1562C4849B5616FC55F94AF27={InputName=76EFF2F1562C4849B5616FC55F94AF27}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:20,316 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 76EFF2F1562C4849B5616FC55F94AF27, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:20,316 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075]
2016-05-18 02:07:20,317 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075]
2016-05-18 02:07:20,317 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075]
2016-05-18 02:07:20,317 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.-1_VALUES_ALL, applicationId=application_1463537240145_0001, dagName=[C3505A0A497547CF9534C174CF50AF0B/E66FE1A2BFAD416B9DBE47CD4705FC18] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-18 02:07:20,317 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075]
2016-05-18 02:07:20,318 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:20,318 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 76EFF2F1562C4849B5616FC55F94AF27 on vertex vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075]
2016-05-18 02:07:20,321 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 76EFF2F1562C4849B5616FC55F94AF27 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:20,324 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:20,324 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:20,325 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:20,325 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:20,325 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 76EFF2F1562C4849B5616FC55F94AF27 on vertex vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075]
2016-05-18 02:07:20,326 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:20,326 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {76EFF2F1562C4849B5616FC55F94AF27=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:20,327 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075] parallelism set to 1
2016-05-18 02:07:20,327 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:20,327 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075], numAdditionalOutputs=1
2016-05-18 02:07:20,328 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=E33CC2F9112B46D08806D61487F0F0A4, vertex=vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:20,329 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D9231161C767483CB157AB7C4DE55075:E33CC2F9112B46D08806D61487F0F0A4 using oldmapred API
2016-05-18 02:07:20,329 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=E33CC2F9112B46D08806D61487F0F0A4, vertexName=D9231161C767483CB157AB7C4DE55075, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:20,330 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240145_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D9231161C767483CB157AB7C4DE55075, vertexId=vertex_1463537240145_0001_1_00, initRequestedTime=1463537240316, initedTime=1463537240327, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:20,330 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075] with distanceFromRoot: 0
2016-05-18 02:07:20,330 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:20,331 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240145_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537240145_0001_1_00, startRequestedTime=1463537240318, startedTime=1463537240330
2016-05-18 02:07:20,331 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240145_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537240145_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:20,331 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:20,331 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in D9231161C767483CB157AB7C4DE55075
2016-05-18 02:07:20,332 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075]
2016-05-18 02:07:20,333 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240145_0001_1][Event:TASK_STARTED]: vertexName=D9231161C767483CB157AB7C4DE55075, taskId=task_1463537240145_0001_1_00_000000, scheduledTime=1463537240333, launchTime=1463537240333
2016-05-18 02:07:20,333 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075]
2016-05-18 02:07:20,334 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:20,336 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537240145_0001_00_000001, launchTime=1463537240336
2016-05-18 02:07:20,336 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537240145_0001_00_000001
2016-05-18 02:07:20,340 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537240145_0001_1_00_000000_0] started. Is using containerId: [container_1463537240145_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:20,340 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537240145_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537240145_0001_1_00_000000_0
2016-05-18 02:07:20 Starting to run new task attempt: attempt_1463537240145_0001_1_00_000000_0
2016-05-18 02:07:20,340 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240145_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D9231161C767483CB157AB7C4DE55075, taskAttemptId=attempt_1463537240145_0001_1_00_000000_0, startTime=1463537240339, containerId=container_1463537240145_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:20,340 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537240145_0001_1_00_000000_0
2016-05-18 02:07:20,341 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:20,341 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:20,341 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C3505A0A497547CF9534C174CF50AF0B/E66FE1A2BFAD416B9DBE47CD4705FC18] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL, VertexName: D9231161C767483CB157AB7C4DE55075, VertexParallelism: 1, TaskAttemptID:attempt_1463537240145_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=76EFF2F1562C4849B5616FC55F94AF27, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E33CC2F9112B46D08806D61487F0F0A4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:20,342 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:20,342 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537240145_0001_1_00_000000_0
2016-05-18 02:07:20,342 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537240145_0001, with dag id: dag_1463537240145_0001_1
2016-05-18 02:07:20,343 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537240145_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:20,344 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537240145_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:20,348 INFO input.MRInput (MRInput.java:initialize(440)) - 76EFF2F1562C4849B5616FC55F94AF27 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:20,348 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 76EFF2F1562C4849B5616FC55F94AF27
2016-05-18 02:07:20,350 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:20,360 INFO output.MROutput (MROutput.java:initialize(421)) - E33CC2F9112B46D08806D61487F0F0A4: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:20,361 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D9231161C767483CB157AB7C4DE55075, ordinal: 0
2016-05-18 02:07:20,361 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D9231161C767483CB157AB7C4DE55075, mem on start (mb), free: 249, total: 385, max: 672
2016-05-18 02:07:20,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:20,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:20,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:20,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:20,361 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[76EFF2F1562C4849B5616FC55F94AF27:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [E33CC2F9112B46D08806D61487F0F0A4:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:20,361 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:20,361 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:20,362 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:20,362 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[76EFF2F1562C4849B5616FC55F94AF27:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [E33CC2F9112B46D08806D61487F0F0A4:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:20,362 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:20,362 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 76EFF2F1562C4849B5616FC55F94AF27 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:20,362 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 76EFF2F1562C4849B5616FC55F94AF27
2016-05-18 02:07:20,362 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:20,362 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:20,362 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:20,363 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537240145_0001_1_00_000000_0
2016-05-18 02:07:20,366 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:20,366 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 76EFF2F1562C4849B5616FC55F94AF27 initialized RecordReader from event
2016-05-18 02:07:20,367 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:20,367 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 76EFF2F1562C4849B5616FC55F94AF27
2016-05-18 02:07:20,367 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_ALL"], id: E33CC2F9112B46D08806D61487F0F0A4
2016-05-18 02:07:20,368 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_ALL"]
2016-05-18 02:07:20,368 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:20,368 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D9231161C767483CB157AB7C4DE55075, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:20,368 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 02:07:20,369 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 02:07:20,370 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 02:07:20,373 INFO output.MROutput (MROutput.java:close(526)) - E33CC2F9112B46D08806D61487F0F0A4 closed
2016-05-18 02:07:20,375 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537240145_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:20,376 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537240145_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:20,376 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537240145_0001_1_00_000000_0: Counters: 8 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463537240368, Process_Duration=8, Process_End_Time=1463537240376, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-18 02:07:20,377 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(817)) - Joining on EventRouter
2016-05-18 02:07:20,377 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D9231161C767483CB157AB7C4DE55075, mem on close (mb), free: 246, total: 385, max: 672
2016-05-18 02:07:20,377 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(851)) - Closed processor for vertex=D9231161C767483CB157AB7C4DE55075, index=0, interruptedStatus=false
2016-05-18 02:07:20,377 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(887)) - Closed input for vertex=D9231161C767483CB157AB7C4DE55075, sourceVertex=76EFF2F1562C4849B5616FC55F94AF27, interruptedStatus=true
2016-05-18 02:07:20,377 INFO output.MROutput (MROutput.java:close(526)) - E33CC2F9112B46D08806D61487F0F0A4 closed
2016-05-18 02:07:20,378 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(911)) - Closed input for vertex=D9231161C767483CB157AB7C4DE55075, sourceVertex=E33CC2F9112B46D08806D61487F0F0A4, interruptedStatus=true
2016-05-18 02:07:20,379 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537240145_0001_1_00_000000_0 : TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 02:07:20,379 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 02:07:20,379 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537240145_0001_00_000001
2016-05-18 02:07:20,379 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240145_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D9231161C767483CB157AB7C4DE55075, taskAttemptId=attempt_1463537240145_0001_1_00_000000_0, creationTime=1463537240333, allocationTime=1463537240335, startTime=1463537240339, finishTime=1463537240379, timeTaken=40, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task: attempt_1463537240145_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
, counters=Counters: 9, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463537240368, Process_Duration=8, Process_End_Time=1463537240376, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-18 02:07:20,379 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537240145_0001_00_000001
2016-05-18 02:07:20,380 INFO impl.TaskImpl (TaskImpl.java:transition(1170)) - Failing task: task_1463537240145_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-18 02:07:20,380 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240145_0001_1][Event:TASK_FINISHED]: vertexName=D9231161C767483CB157AB7C4DE55075, taskId=task_1463537240145_0001_1_00_000000, startTime=1463537240339, finishTime=1463537240380, timeTaken=41, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463537240145_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
], counters=Counters: 0
2016-05-18 02:07:20,381 INFO impl.VertexImpl (VertexImpl.java:transition(3353)) - Failing vertex: vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075] because task failed: task_1463537240145_0001_1_00_000000
2016-05-18 02:07:20,382 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(2102)) - Killing tasks in vertex: vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075] due to trigger: OWN_TASK_FAILURE
2016-05-18 02:07:20,382 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075], tasks=1, failed=1, killed=0, success=0, completed=1, commits=0, err=OWN_TASK_FAILURE
2016-05-18 02:07:20,382 INFO impl.VertexImpl (VertexImpl.java:finishWithTerminationCause(2084)) - Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
2016-05-18 02:07:20,382 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3087)) - Invoking committer abort for vertex, vertexId=vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075]
2016-05-18 02:07:20,380 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537240145_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, errorMessage='TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation'}
2016-05-18 02:07:20,383 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(294)) - Container: container_1463537240145_0001_00_000001 completed but with errors
2016-05-18 02:07:20,382 INFO impl.VertexImpl (VertexImpl.java:run(3095)) - Invoking committer abort for output=E33CC2F9112B46D08806D61487F0F0A4, vertexId=vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075]
2016-05-18 02:07:20,386 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240145_0001_1][Event:VERTEX_FINISHED]: vertexName=D9231161C767483CB157AB7C4DE55075, vertexId=vertex_1463537240145_0001_1_00, initRequestedTime=1463537240316, initedTime=1463537240327, startRequestedTime=1463537240318, startedTime=1463537240330, finishTime=1463537240385, timeTaken=55, status=FAILED, diagnostics=Task failed, taskId=task_1463537240145_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463537240145_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]]
Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
Vertex vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075] killed/failed due to:OWN_TASK_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=1463537240339, firstTasksToStart=[ task_1463537240145_0001_1_00_000000 ], lastTaskFinishTime=1463537240379, lastTasksToFinish=[ task_1463537240145_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-18 02:07:20,386 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-18 02:07:20,387 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240145_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537240145_0001_00_000001, stoppedTime=1463537240387, exitStatus=-1000
2016-05-18 02:07:20,387 INFO node.AMNodeImpl (AMNodeImpl.java:transition(298)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463537240145_0001_1_00_000000_0 failed: true container: container_1463537240145_0001_00_000001 numFailedTAs: 0
2016-05-18 02:07:20,387 INFO container.AMContainerImpl (AMContainerImpl.java:transition(993)) - Container container_1463537240145_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
2016-05-18 02:07:20,387 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-18 02:07:20,388 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=VERTEX_FAILURE
2016-05-18 02:07:20,388 INFO impl.DAGImpl (DAGImpl.java:finishWithTerminationCause(1350)) - DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
2016-05-18 02:07:20,390 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537240145_0001_1, queueSize=0
2016-05-18 02:07:20,391 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240145_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537240145_0001_1, startTime=1463537240315, finishTime=1463537240389, timeTaken=74, status=FAILED, diagnostics=Vertex failed, vertexName=D9231161C767483CB157AB7C4DE55075, vertexId=vertex_1463537240145_0001_1_00, diagnostics=[Task failed, taskId=task_1463537240145_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463537240145_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075] killed/failed due to:OWN_TASK_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0, counters=Counters: 4, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, AM_CPU_MILLISECONDS=140, AM_GC_TIME_MILLIS=0
2016-05-18 02:07:20,391 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3082)) - Ignoring multiple aborts for vertex: vertex_1463537240145_0001_1_00 [D9231161C767483CB157AB7C4DE55075]
2016-05-18 02:07:20,392 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537240145_0001_1 finished with state: FAILED
2016-05-18 02:07:20,392 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537240145_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:20,392 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(260)) - Ignoring stop request for containerId: container_1463537240145_0001_00_000001
2016-05-18 02:07:20 Completed Dag: dag_1463537240145_0001_1
2016-05-18 02:07:20,393 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537240145_0001_1, dagState=FAILED
2016-05-18 02:07:20,393 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537240145_0001_1_post
2016-05-18 02:07:20,394 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:20,394 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 1
2016-05-18 02:07:20,394 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:20,394 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[C3505A0A497547CF9534C174CF50AF0B/E66FE1A2BFAD416B9DBE47CD4705FC18] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL, with id=dag_1463537240145_0001_1
2016-05-18 02:07:20,394 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[C3505A0A497547CF9534C174CF50AF0B/E66FE1A2BFAD416B9DBE47CD4705FC18] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL, with id=dag_1463537240145_0001_1
2016-05-18 02:07:20,397 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ld'_RESULTS.-1_VALUES_ALL, applicationId=application_1463537240145_0001
2016-05-18 02:07:20,400 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:20,400 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:20,400 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:20,400 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:20,400 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:20,401 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:20,401 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:20,402 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:20,402 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:20,402 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:20,402 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:20,402 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:20,401 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:20,403 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:20,407 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/C3505A0A497547CF9534C174CF50AF0B/E66FE1A2BFAD416B9DBE47CD4705FC18/.tez/application_1463537240145_0001
2016-05-18 02:07:20,407 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-18 02:07:20,408 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-18 02:07:20,408 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-18 02:07:20,409 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_ALL/_temporary
2016-05-18 02:07:20,409 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.285
2016-05-18 02:07:20,409 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_VALUES_ALL
2016-05-18 02:07:20,447 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:20,448 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:20,488 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 02:07:20,488 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 02:07:20,490 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.006
2016-05-18 02:07:20,493 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.009
2016-05-18 02:07:20,493 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 02:07:20,494 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 02:07:20,494 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_ARGS_'field2'
2016-05-18 02:07:20,522 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:20,522 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:20,583 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-18 02:07:20,594 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-05-18 02:07:20,594 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:20,594 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:20,595 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:20,622 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:20,623 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_ARGS_RESULTS
2016-05-18 02:07:20,640 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:20,641 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:20,641 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_ARGS_RESULTS"]
2016-05-18 02:07:20,641 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:20,641 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:20,642 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:20,642 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...'_RESULTS.-1_ARGS_RESULTS
2016-05-18 02:07:20,643 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:20,662 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:20,666 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:20,667 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:20,667 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:20,676 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/215132201A4D4650BE6CDF675EA0CC76/E5E5EDC365904AE2BB8AE29A74465C2C/.tez/application_1463537240667_0001 doesn't exist and is created
2016-05-18 02:07:20,696 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:20,697 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:20,698 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/215132201A4D4650BE6CDF675EA0CC76/E5E5EDC365904AE2BB8AE29A74465C2C/.tez/application_1463537240667_0001_wd
2016-05-18 02:07:20,715 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537240667_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:20,716 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:20,716 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:20,717 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:20,718 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:20,718 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:20,718 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:20,719 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:20,721 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:20,721 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:20,721 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:20,721 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:20,722 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:20,722 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:20,722 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:20,722 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:20,722 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:20,723 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:20,723 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:20,723 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:20,725 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:20,725 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:20,726 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:20,726 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/215132201A4D4650BE6CDF675EA0CC76/E5E5EDC365904AE2BB8AE29A74465C2C/.tez/application_1463537240667_0001_wd/localmode-log-dir
2016-05-18 02:07:20,726 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/215132201A4D4650BE6CDF675EA0CC76/E5E5EDC365904AE2BB8AE29A74465C2C/.tez/application_1463537240667_0001_wd/localmode-log-dir/history.txt.appattempt_1463537240667_0001_000000, maxErrors=10
2016-05-18 02:07:20,727 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/215132201A4D4650BE6CDF675EA0CC76/E5E5EDC365904AE2BB8AE29A74465C2C/.tez/application_1463537240667_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:20,727 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537240667_0001_000000, appSubmitTime=1463537240714, launchTime=1463537240715
2016-05-18 02:07:20,732 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:20,735 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:59243
2016-05-18 02:07:20,736 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537240667
2016-05-18 02:07:20,737 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537240667_0001_000000, startTime=1463537240737
2016-05-18 02:07:20,738 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:20,797 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:20,797 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:20,798 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...'_RESULTS.-1_ARGS_RESULTS, applicationId=application_1463537240667_0001, dagName=[215132201A4D4650BE6CDF675EA0CC76/E5E5EDC365904AE2BB8AE29A74465C2C] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS
2016-05-18 02:07:20,817 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [215132201A4D4650BE6CDF675EA0CC76/E5E5EDC365904AE2BB8AE29A74465C2C] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS
2016-05-18 02:07:20,818 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:20,819 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537240667_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/215132201A4D4650BE6CDF675EA0CC76/E5E5EDC365904AE2BB8AE29A74465C2C/.tez/application_1463537240667_0001_wd/localmode-log-dir/dag_1463537240667_0001_1.dot
2016-05-18 02:07:20,819 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/215132201A4D4650BE6CDF675EA0CC76/E5E5EDC365904AE2BB8AE29A74465C2C/.tez/application_1463537240667_0001_wd/localmode-log-dir/dag_1463537240667_0001_1-tez-dag.pb.txt
2016-05-18 02:07:20,823 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537240667_0001_1
2016-05-18 02:07:20,823 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:20,823 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [215132201A4D4650BE6CDF675EA0CC76/E5E5EDC365904AE2BB8AE29A74465C2C] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS
2016-05-18 02:07:20 Running Dag: dag_1463537240667_0001_1
2016-05-18 02:07:20,840 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240667_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537240667_0001_1, submitTime=1463537240818
2016-05-18 02:07:20,841 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/215132201A4D4650BE6CDF675EA0CC76/E5E5EDC365904AE2BB8AE29A74465C2C/.tez/application_1463537240667_0001/tez-conf.pb" } size: 40947 timestamp: 1463537240000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:20,841 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7]
2016-05-18 02:07:20,842 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7]
2016-05-18 02:07:20,842 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:20,842 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:20,843 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240667_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537240667_0001_1, initTime=1463537240842
2016-05-18 02:07:20,843 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537240667_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:20,844 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:20,844 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240667_0001_1][Event:DAG_STARTED]: dagID=dag_1463537240667_0001_1, startTime=1463537240844
2016-05-18 02:07:20,844 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...'_RESULTS.-1_ARGS_RESULTS, applicationId=application_1463537240667_0001, dagName=[215132201A4D4650BE6CDF675EA0CC76/E5E5EDC365904AE2BB8AE29A74465C2C] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS
2016-05-18 02:07:20,844 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537240667_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:20,845 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: A1211FEB16094D8E854DCD009028CBD7 : {4F03EDA878754A8A9E4CB2A3AA463BC7={InputName=4F03EDA878754A8A9E4CB2A3AA463BC7}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:20,845 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 4F03EDA878754A8A9E4CB2A3AA463BC7, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:20,845 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7]
2016-05-18 02:07:20,845 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7]
2016-05-18 02:07:20,845 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7]
2016-05-18 02:07:20,845 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7]
2016-05-18 02:07:20,846 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:20,846 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 4F03EDA878754A8A9E4CB2A3AA463BC7 on vertex vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7]
2016-05-18 02:07:20,849 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 4F03EDA878754A8A9E4CB2A3AA463BC7 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:20,853 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:20,853 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:20,853 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:20,854 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:20,854 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 4F03EDA878754A8A9E4CB2A3AA463BC7 on vertex vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7]
2016-05-18 02:07:20,855 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:20,855 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {4F03EDA878754A8A9E4CB2A3AA463BC7=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:20,856 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7] parallelism set to 1
2016-05-18 02:07:20,856 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:20,856 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7], numAdditionalOutputs=1
2016-05-18 02:07:20,856 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=EF7EB89458FF43F09F6C44200BEA760D, vertex=vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:20,859 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for A1211FEB16094D8E854DCD009028CBD7:EF7EB89458FF43F09F6C44200BEA760D using oldmapred API
2016-05-18 02:07:20,859 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=EF7EB89458FF43F09F6C44200BEA760D, vertexName=A1211FEB16094D8E854DCD009028CBD7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:20,860 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240667_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A1211FEB16094D8E854DCD009028CBD7, vertexId=vertex_1463537240667_0001_1_00, initRequestedTime=1463537240845, initedTime=1463537240856, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:20,860 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7] with distanceFromRoot: 0
2016-05-18 02:07:20,860 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:20,861 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240667_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537240667_0001_1_00, startRequestedTime=1463537240846, startedTime=1463537240860
2016-05-18 02:07:20,861 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240667_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537240667_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:20,861 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:20,862 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in A1211FEB16094D8E854DCD009028CBD7
2016-05-18 02:07:20,862 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7]
2016-05-18 02:07:20,863 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240667_0001_1][Event:TASK_STARTED]: vertexName=A1211FEB16094D8E854DCD009028CBD7, taskId=task_1463537240667_0001_1_00_000000, scheduledTime=1463537240862, launchTime=1463537240862
2016-05-18 02:07:20,863 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7]
2016-05-18 02:07:20,864 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:20,866 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537240667_0001_00_000001, launchTime=1463537240866
2016-05-18 02:07:20,866 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537240667_0001_00_000001
2016-05-18 02:07:20,867 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537240667_0001, with dag id: dag_1463537240667_0001_1
2016-05-18 02:07:20,867 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537240667_0001_1_00_000000_0] started. Is using containerId: [container_1463537240667_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:20,867 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240667_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A1211FEB16094D8E854DCD009028CBD7, taskAttemptId=attempt_1463537240667_0001_1_00_000000_0, startTime=1463537240867, containerId=container_1463537240667_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:20,868 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537240667_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537240667_0001_1_00_000000_0
2016-05-18 02:07:20 Starting to run new task attempt: attempt_1463537240667_0001_1_00_000000_0
2016-05-18 02:07:20,869 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537240667_0001_1_00_000000_0
2016-05-18 02:07:20,869 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:20,869 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:20,871 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [215132201A4D4650BE6CDF675EA0CC76/E5E5EDC365904AE2BB8AE29A74465C2C] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS, VertexName: A1211FEB16094D8E854DCD009028CBD7, VertexParallelism: 1, TaskAttemptID:attempt_1463537240667_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4F03EDA878754A8A9E4CB2A3AA463BC7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EF7EB89458FF43F09F6C44200BEA760D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:20,871 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:20,872 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537240667_0001_1_00_000000_0
2016-05-18 02:07:20,873 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537240667_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:20,874 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537240667_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:20,876 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:20,878 INFO input.MRInput (MRInput.java:initialize(440)) - 4F03EDA878754A8A9E4CB2A3AA463BC7 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:20,878 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 4F03EDA878754A8A9E4CB2A3AA463BC7
2016-05-18 02:07:20,885 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A1211FEB16094D8E854DCD009028CBD7, ordinal: 0
2016-05-18 02:07:20,885 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A1211FEB16094D8E854DCD009028CBD7, mem on start (mb), free: 156, total: 385, max: 672
2016-05-18 02:07:20,885 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:20,885 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:20,885 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:20,889 INFO output.MROutput (MROutput.java:initialize(421)) - EF7EB89458FF43F09F6C44200BEA760D: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:20,889 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:20,889 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[4F03EDA878754A8A9E4CB2A3AA463BC7:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [EF7EB89458FF43F09F6C44200BEA760D:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:20,889 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:20,889 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:20,889 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:20,890 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[4F03EDA878754A8A9E4CB2A3AA463BC7:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [EF7EB89458FF43F09F6C44200BEA760D:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:20,890 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:20,890 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 4F03EDA878754A8A9E4CB2A3AA463BC7 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:20,890 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:20,890 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:20,891 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 4F03EDA878754A8A9E4CB2A3AA463BC7
2016-05-18 02:07:20,891 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:20,891 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537240667_0001_1_00_000000_0
2016-05-18 02:07:20,894 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:20,894 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:20,894 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 4F03EDA878754A8A9E4CB2A3AA463BC7 initialized RecordReader from event
2016-05-18 02:07:20,894 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 4F03EDA878754A8A9E4CB2A3AA463BC7
2016-05-18 02:07:20,895 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_ARGS_RESULTS"], id: EF7EB89458FF43F09F6C44200BEA760D
2016-05-18 02:07:20,895 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_ARGS_RESULTS"]
2016-05-18 02:07:20,895 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:20,895 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A1211FEB16094D8E854DCD009028CBD7, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:20,898 INFO output.MROutput (MROutput.java:close(526)) - EF7EB89458FF43F09F6C44200BEA760D closed
2016-05-18 02:07:20,899 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537240667_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:20,900 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537240667_0001_1_00_000000_0
2016-05-18 02:07:20,900 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A1211FEB16094D8E854DCD009028CBD7, mem on close (mb), free: 154, total: 385, max: 672
2016-05-18 02:07:20,901 INFO output.MROutput (MROutput.java:close(526)) - EF7EB89458FF43F09F6C44200BEA760D closed
2016-05-18 02:07:20,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:20 Completed running task attempt: attempt_1463537240667_0001_1_00_000000_0
2016-05-18 02:07:20,902 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537240667_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:20,902 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537240667_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:20,904 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537240667_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537240895, Process_Duration=5, Process_End_Time=1463537240900, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:20,905 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537240667_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:20,905 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:20,905 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:20,905 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:20,905 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240667_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A1211FEB16094D8E854DCD009028CBD7, taskAttemptId=attempt_1463537240667_0001_1_00_000000_0, creationTime=1463537240863, allocationTime=1463537240865, startTime=1463537240867, finishTime=1463537240905, timeTaken=38, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:20,906 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537240667_0001_1_00_000000_0
2016-05-18 02:07:20,906 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537240667_0001_00_000001
2016-05-18 02:07:20,907 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240667_0001_1][Event:TASK_FINISHED]: vertexName=A1211FEB16094D8E854DCD009028CBD7, taskId=task_1463537240667_0001_1_00_000000, startTime=1463537240867, finishTime=1463537240906, timeTaken=39, status=SUCCEEDED, successfulAttemptID=attempt_1463537240667_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537240895, Process_Duration=5, Process_End_Time=1463537240900, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:20,908 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:20,908 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7]
2016-05-18 02:07:20,909 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240667_0001_1][Event:VERTEX_FINISHED]: vertexName=A1211FEB16094D8E854DCD009028CBD7, vertexId=vertex_1463537240667_0001_1_00, initRequestedTime=1463537240845, initedTime=1463537240856, startRequestedTime=1463537240846, startedTime=1463537240860, finishTime=1463537240908, timeTaken=48, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537240895, Process_Duration=5, Process_End_Time=1463537240900, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537240867, firstTasksToStart=[ task_1463537240667_0001_1_00_000000 ], lastTaskFinishTime=1463537240905, lastTasksToFinish=[ task_1463537240667_0001_1_00_000000 ], minTaskDuration=38, maxTaskDuration=38, avgTaskDuration=38.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537240667_0001_1_00_000000 ], longestDurationTasks=[ task_1463537240667_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:20,909 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:20,909 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240667_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537240667_0001_00_000001, stoppedTime=1463537240909, exitStatus=0
2016-05-18 02:07:20,910 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:20,910 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:20,910 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537240667_0001_1
2016-05-18 02:07:20,910 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240667_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537240667_0001_1
2016-05-18 02:07:20,910 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537240667_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:20,910 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: EF7EB89458FF43F09F6C44200BEA760D for vertex: vertex_1463537240667_0001_1_00 [A1211FEB16094D8E854DCD009028CBD7], outputName: EF7EB89458FF43F09F6C44200BEA760D
2016-05-18 02:07:20,910 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537240667_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:20,913 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537240667_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:20,913 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537240667_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:20,914 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537240667_0001_00_000001, Exiting
2016-05-18 02:07:20,914 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537240667_0001_00_000001
2016-05-18 02:07:20,914 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537240667_0001_00_000001
2016-05-18 02:07:20,915 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537240667_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:20,915 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537240667_0001_00_000001 completed successfully
2016-05-18 02:07:20,922 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:EF7EB89458FF43F09F6C44200BEA760D of vertex/vertexGroup:A1211FEB16094D8E854DCD009028CBD7 isVertexGroupOutput:false
2016-05-18 02:07:20,922 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:20,923 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537240667_0001_1, queueSize=0
2016-05-18 02:07:20,924 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537240667_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537240667_0001_1, startTime=1463537240844, finishTime=1463537240922, timeTaken=78, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=140, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537240895, Process_Duration=5, Process_End_Time=1463537240900, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:20,924 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537240667_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:20,924 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537240667_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:20 Completed Dag: dag_1463537240667_0001_1
2016-05-18 02:07:20,924 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537240667_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:20,925 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537240667_0001_1_post
2016-05-18 02:07:20,925 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:20,925 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:20,925 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:20,925 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[215132201A4D4650BE6CDF675EA0CC76/E5E5EDC365904AE2BB8AE29A74465C2C] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS, with id=dag_1463537240667_0001_1
2016-05-18 02:07:20,925 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[215132201A4D4650BE6CDF675EA0CC76/E5E5EDC365904AE2BB8AE29A74465C2C] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS, with id=dag_1463537240667_0001_1
2016-05-18 02:07:20,931 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...'_RESULTS.-1_ARGS_RESULTS, applicationId=application_1463537240667_0001
2016-05-18 02:07:20,934 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:20,934 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:20,935 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:20,935 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:20,935 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:20,936 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:20,936 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:20,936 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:20,937 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:20,937 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:20,938 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:20,938 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:20,939 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:20,940 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:20,943 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/215132201A4D4650BE6CDF675EA0CC76/E5E5EDC365904AE2BB8AE29A74465C2C/.tez/application_1463537240667_0001
2016-05-18 02:07:20,943 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_ARGS_RESULTS/_temporary
2016-05-18 02:07:20,944 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.303
2016-05-18 02:07:21,037 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_ARGS_RESULTS
2016-05-18 02:07:21,065 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:21,065 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:21,093 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 02:07:21,093 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 02:07:21,098 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-05-18 02:07:21,100 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.006
2016-05-18 02:07:21,101 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 02:07:21,101 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 02:07:21,102 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_ARGS_ALL
2016-05-18 02:07:21,130 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:21,130 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:21,172 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-18 02:07:21,181 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-05-18 02:07:21,182 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:21,182 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:21,182 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:21,210 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:21,211 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_-1
2016-05-18 02:07:21,229 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:21,229 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:21,229 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_-1"]
2016-05-18 02:07:21,230 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:21,230 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:21,230 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:21,231 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ld'_RESULTS.0_'field2'_-1
2016-05-18 02:07:21,231 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:21,252 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:21,257 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:21,258 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:21,258 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:21,269 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/354C41045A934DB5A1AD24A9F1514F78/F5226A57B3B24D50A7A2768DEC3CFBF6/.tez/application_1463537241257_0001 doesn't exist and is created
2016-05-18 02:07:21,290 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:21,290 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:21,291 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/354C41045A934DB5A1AD24A9F1514F78/F5226A57B3B24D50A7A2768DEC3CFBF6/.tez/application_1463537241257_0001_wd
2016-05-18 02:07:21,306 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537241257_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:21,307 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:21,307 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:21,309 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:21,309 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:21,309 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:21,309 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:21,310 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:21,312 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:21,312 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:21,312 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:21,312 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:21,313 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:21,313 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:21,313 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:21,313 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:21,313 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:21,313 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:21,314 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:21,314 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:21,317 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:21,318 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:21,318 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:21,319 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/354C41045A934DB5A1AD24A9F1514F78/F5226A57B3B24D50A7A2768DEC3CFBF6/.tez/application_1463537241257_0001_wd/localmode-log-dir
2016-05-18 02:07:21,319 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/354C41045A934DB5A1AD24A9F1514F78/F5226A57B3B24D50A7A2768DEC3CFBF6/.tez/application_1463537241257_0001_wd/localmode-log-dir/history.txt.appattempt_1463537241257_0001_000000, maxErrors=10
2016-05-18 02:07:21,319 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/354C41045A934DB5A1AD24A9F1514F78/F5226A57B3B24D50A7A2768DEC3CFBF6/.tez/application_1463537241257_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:21,320 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537241257_0001_000000, appSubmitTime=1463537241306, launchTime=1463537241306
2016-05-18 02:07:21,323 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:21,327 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:42100
2016-05-18 02:07:21,327 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537241257
2016-05-18 02:07:21,331 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537241257_0001_000000, startTime=1463537241331
2016-05-18 02:07:21,332 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:21,390 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:21,391 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:21,391 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.0_'field2'_-1, applicationId=application_1463537241257_0001, dagName=[354C41045A934DB5A1AD24A9F1514F78/F5226A57B3B24D50A7A2768DEC3CFBF6] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1
2016-05-18 02:07:21,410 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [354C41045A934DB5A1AD24A9F1514F78/F5226A57B3B24D50A7A2768DEC3CFBF6] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1
2016-05-18 02:07:21,411 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:21,412 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537241257_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/354C41045A934DB5A1AD24A9F1514F78/F5226A57B3B24D50A7A2768DEC3CFBF6/.tez/application_1463537241257_0001_wd/localmode-log-dir/dag_1463537241257_0001_1.dot
2016-05-18 02:07:21,412 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/354C41045A934DB5A1AD24A9F1514F78/F5226A57B3B24D50A7A2768DEC3CFBF6/.tez/application_1463537241257_0001_wd/localmode-log-dir/dag_1463537241257_0001_1-tez-dag.pb.txt
2016-05-18 02:07:21,416 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537241257_0001_1
2016-05-18 02:07:21,416 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:21,417 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [354C41045A934DB5A1AD24A9F1514F78/F5226A57B3B24D50A7A2768DEC3CFBF6] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1
2016-05-18 02:07:21 Running Dag: dag_1463537241257_0001_1
2016-05-18 02:07:21,433 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241257_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537241257_0001_1, submitTime=1463537241411
2016-05-18 02:07:21,434 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/354C41045A934DB5A1AD24A9F1514F78/F5226A57B3B24D50A7A2768DEC3CFBF6/.tez/application_1463537241257_0001/tez-conf.pb" } size: 40947 timestamp: 1463537241000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:21,434 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303]
2016-05-18 02:07:21,435 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303]
2016-05-18 02:07:21,435 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:21,435 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:21,436 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241257_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537241257_0001_1, initTime=1463537241436
2016-05-18 02:07:21,436 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537241257_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:21,437 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:21,437 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241257_0001_1][Event:DAG_STARTED]: dagID=dag_1463537241257_0001_1, startTime=1463537241437
2016-05-18 02:07:21,437 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537241257_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:21,438 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: C35A0F525AE947D5B461341F721FA303 : {FDF4057EEB134EA097CAE29F901B3A6D={InputName=FDF4057EEB134EA097CAE29F901B3A6D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:21,438 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: FDF4057EEB134EA097CAE29F901B3A6D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:21,438 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303]
2016-05-18 02:07:21,438 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.0_'field2'_-1, applicationId=application_1463537241257_0001, dagName=[354C41045A934DB5A1AD24A9F1514F78/F5226A57B3B24D50A7A2768DEC3CFBF6] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1
2016-05-18 02:07:21,438 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303]
2016-05-18 02:07:21,438 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303]
2016-05-18 02:07:21,438 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303]
2016-05-18 02:07:21,439 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:21,439 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: FDF4057EEB134EA097CAE29F901B3A6D on vertex vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303]
2016-05-18 02:07:21,442 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input FDF4057EEB134EA097CAE29F901B3A6D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:21,444 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:21,444 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:21,444 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:21,444 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:21,445 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: FDF4057EEB134EA097CAE29F901B3A6D on vertex vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303]
2016-05-18 02:07:21,447 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:21,447 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {FDF4057EEB134EA097CAE29F901B3A6D=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:21,448 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303] parallelism set to 1
2016-05-18 02:07:21,449 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:21,450 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303], numAdditionalOutputs=1
2016-05-18 02:07:21,450 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=0934DC505B364D05B9FBBCF22A56E9D9, vertex=vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:21,452 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for C35A0F525AE947D5B461341F721FA303:0934DC505B364D05B9FBBCF22A56E9D9 using oldmapred API
2016-05-18 02:07:21,452 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=0934DC505B364D05B9FBBCF22A56E9D9, vertexName=C35A0F525AE947D5B461341F721FA303, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:21,453 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241257_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C35A0F525AE947D5B461341F721FA303, vertexId=vertex_1463537241257_0001_1_00, initRequestedTime=1463537241437, initedTime=1463537241450, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:21,453 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303] with distanceFromRoot: 0
2016-05-18 02:07:21,454 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:21,454 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241257_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537241257_0001_1_00, startRequestedTime=1463537241439, startedTime=1463537241454
2016-05-18 02:07:21,454 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241257_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537241257_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:21,455 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:21,455 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in C35A0F525AE947D5B461341F721FA303
2016-05-18 02:07:21,455 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303]
2016-05-18 02:07:21,456 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241257_0001_1][Event:TASK_STARTED]: vertexName=C35A0F525AE947D5B461341F721FA303, taskId=task_1463537241257_0001_1_00_000000, scheduledTime=1463537241456, launchTime=1463537241456
2016-05-18 02:07:21,456 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303]
2016-05-18 02:07:21,458 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537241257_0001, with dag id: dag_1463537241257_0001_1
2016-05-18 02:07:21,458 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:21,461 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537241257_0001_00_000001, launchTime=1463537241460
2016-05-18 02:07:21,461 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537241257_0001_00_000001
2016-05-18 02:07:21,464 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537241257_0001_1_00_000000_0] started. Is using containerId: [container_1463537241257_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:21,464 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241257_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C35A0F525AE947D5B461341F721FA303, taskAttemptId=attempt_1463537241257_0001_1_00_000000_0, startTime=1463537241463, containerId=container_1463537241257_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:21,465 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537241257_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537241257_0001_1_00_000000_0
2016-05-18 02:07:21 Starting to run new task attempt: attempt_1463537241257_0001_1_00_000000_0
2016-05-18 02:07:21,467 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537241257_0001_1_00_000000_0
2016-05-18 02:07:21,467 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:21,467 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:21,468 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [354C41045A934DB5A1AD24A9F1514F78/F5226A57B3B24D50A7A2768DEC3CFBF6] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1, VertexName: C35A0F525AE947D5B461341F721FA303, VertexParallelism: 1, TaskAttemptID:attempt_1463537241257_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FDF4057EEB134EA097CAE29F901B3A6D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=0934DC505B364D05B9FBBCF22A56E9D9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:21,468 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:21,468 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537241257_0001_1_00_000000_0
2016-05-18 02:07:21,469 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537241257_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:21,470 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537241257_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:21,475 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:21,475 INFO input.MRInput (MRInput.java:initialize(440)) - FDF4057EEB134EA097CAE29F901B3A6D using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:21,476 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: FDF4057EEB134EA097CAE29F901B3A6D
2016-05-18 02:07:21,486 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C35A0F525AE947D5B461341F721FA303, ordinal: 0
2016-05-18 02:07:21,486 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C35A0F525AE947D5B461341F721FA303, mem on start (mb), free: 255, total: 386, max: 672
2016-05-18 02:07:21,486 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:21,486 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:21,486 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:21,491 INFO output.MROutput (MROutput.java:initialize(421)) - 0934DC505B364D05B9FBBCF22A56E9D9: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:21,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:21,491 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[FDF4057EEB134EA097CAE29F901B3A6D:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [0934DC505B364D05B9FBBCF22A56E9D9:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:21,492 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:21,492 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:21,492 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:21,492 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[FDF4057EEB134EA097CAE29F901B3A6D:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [0934DC505B364D05B9FBBCF22A56E9D9:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:21,492 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:21,492 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: FDF4057EEB134EA097CAE29F901B3A6D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:21,492 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: FDF4057EEB134EA097CAE29F901B3A6D
2016-05-18 02:07:21,493 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:21,493 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:21,493 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:21,493 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537241257_0001_1_00_000000_0
2016-05-18 02:07:21,496 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:21,496 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - FDF4057EEB134EA097CAE29F901B3A6D initialized RecordReader from event
2016-05-18 02:07:21,497 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:21,498 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: FDF4057EEB134EA097CAE29F901B3A6D
2016-05-18 02:07:21,506 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_-1"], id: 0934DC505B364D05B9FBBCF22A56E9D9
2016-05-18 02:07:21,506 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_-1"]
2016-05-18 02:07:21,507 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:21,507 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C35A0F525AE947D5B461341F721FA303, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:21,510 INFO output.MROutput (MROutput.java:close(526)) - 0934DC505B364D05B9FBBCF22A56E9D9 closed
2016-05-18 02:07:21,510 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537241257_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:21,511 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537241257_0001_1_00_000000_0
2016-05-18 02:07:21,511 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C35A0F525AE947D5B461341F721FA303, mem on close (mb), free: 253, total: 386, max: 672
2016-05-18 02:07:21,512 INFO output.MROutput (MROutput.java:close(526)) - 0934DC505B364D05B9FBBCF22A56E9D9 closed
2016-05-18 02:07:21,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:21 Completed running task attempt: attempt_1463537241257_0001_1_00_000000_0
2016-05-18 02:07:21,513 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537241257_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:21,513 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537241257_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:21,513 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537241257_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537241507, Process_Duration=4, Process_End_Time=1463537241511, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:21,516 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537241257_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:21,516 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:21,516 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241257_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C35A0F525AE947D5B461341F721FA303, taskAttemptId=attempt_1463537241257_0001_1_00_000000_0, creationTime=1463537241456, allocationTime=1463537241460, startTime=1463537241463, finishTime=1463537241516, timeTaken=53, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:21,517 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537241257_0001_1_00_000000_0
2016-05-18 02:07:21,516 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:21,517 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:21,517 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241257_0001_1][Event:TASK_FINISHED]: vertexName=C35A0F525AE947D5B461341F721FA303, taskId=task_1463537241257_0001_1_00_000000, startTime=1463537241463, finishTime=1463537241517, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1463537241257_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537241507, Process_Duration=4, Process_End_Time=1463537241511, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:21,518 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:21,518 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303]
2016-05-18 02:07:21,517 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537241257_0001_00_000001
2016-05-18 02:07:21,519 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241257_0001_1][Event:VERTEX_FINISHED]: vertexName=C35A0F525AE947D5B461341F721FA303, vertexId=vertex_1463537241257_0001_1_00, initRequestedTime=1463537241437, initedTime=1463537241450, startRequestedTime=1463537241439, startedTime=1463537241454, finishTime=1463537241518, timeTaken=64, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537241507, Process_Duration=4, Process_End_Time=1463537241511, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537241463, firstTasksToStart=[ task_1463537241257_0001_1_00_000000 ], lastTaskFinishTime=1463537241516, lastTasksToFinish=[ task_1463537241257_0001_1_00_000000 ], minTaskDuration=53, maxTaskDuration=53, avgTaskDuration=53.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537241257_0001_1_00_000000 ], longestDurationTasks=[ task_1463537241257_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:21,520 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:21,520 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241257_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537241257_0001_00_000001, stoppedTime=1463537241520, exitStatus=0
2016-05-18 02:07:21,520 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:21,520 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:21,520 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537241257_0001_1
2016-05-18 02:07:21,520 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241257_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537241257_0001_1
2016-05-18 02:07:21,520 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537241257_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:21,520 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 0934DC505B364D05B9FBBCF22A56E9D9 for vertex: vertex_1463537241257_0001_1_00 [C35A0F525AE947D5B461341F721FA303], outputName: 0934DC505B364D05B9FBBCF22A56E9D9
2016-05-18 02:07:21,521 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537241257_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:21,529 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537241257_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:21,530 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537241257_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:21,530 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537241257_0001_00_000001, Exiting
2016-05-18 02:07:21,530 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537241257_0001_00_000001
2016-05-18 02:07:21,530 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537241257_0001_00_000001
2016-05-18 02:07:21,531 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537241257_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:21,531 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537241257_0001_00_000001 completed successfully
2016-05-18 02:07:21,533 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:0934DC505B364D05B9FBBCF22A56E9D9 of vertex/vertexGroup:C35A0F525AE947D5B461341F721FA303 isVertexGroupOutput:false
2016-05-18 02:07:21,533 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:21,534 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537241257_0001_1, queueSize=0
2016-05-18 02:07:21,535 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241257_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537241257_0001_1, startTime=1463537241437, finishTime=1463537241533, timeTaken=96, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=190, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537241507, Process_Duration=4, Process_End_Time=1463537241511, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:21,535 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537241257_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:21,535 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537241257_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:21 Completed Dag: dag_1463537241257_0001_1
2016-05-18 02:07:21,535 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537241257_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:21,535 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537241257_0001_1_post
2016-05-18 02:07:21,535 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:21,536 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:21,536 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:21,536 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[354C41045A934DB5A1AD24A9F1514F78/F5226A57B3B24D50A7A2768DEC3CFBF6] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1, with id=dag_1463537241257_0001_1
2016-05-18 02:07:21,536 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[354C41045A934DB5A1AD24A9F1514F78/F5226A57B3B24D50A7A2768DEC3CFBF6] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1, with id=dag_1463537241257_0001_1
2016-05-18 02:07:21,547 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ld'_RESULTS.0_'field2'_-1, applicationId=application_1463537241257_0001
2016-05-18 02:07:21,548 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:21,548 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:21,548 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:21,548 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:21,549 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:21,549 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:21,549 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:21,550 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:21,550 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:21,550 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:21,550 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:21,550 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:21,551 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:21,552 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:21,555 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/354C41045A934DB5A1AD24A9F1514F78/F5226A57B3B24D50A7A2768DEC3CFBF6/.tez/application_1463537241257_0001
2016-05-18 02:07:21,555 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_-1/_temporary
2016-05-18 02:07:21,556 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.326
2016-05-18 02:07:21,636 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_'field2'_-1
2016-05-18 02:07:21,667 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:21,667 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:21,708 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-18 02:07:21,712 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 02:07:21,712 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 02:07:21,716 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-18 02:07:21,717 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:21,717 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:21,717 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:21,736 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:21,737 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_'field2'
2016-05-18 02:07:21,756 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:21,756 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:21,757 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_'field2'"]
2016-05-18 02:07:21,757 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:21,757 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:21,757 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:21,758 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...SULTS.0_'field2'_'field2'
2016-05-18 02:07:21,758 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:21,781 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:21,789 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:21,790 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:21,790 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:21,801 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8F8A5910924A47518721068D266FD66B/F7788F2A05C443E5AC4321B2F87870CE/.tez/application_1463537241790_0001 doesn't exist and is created
2016-05-18 02:07:21,825 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:21,826 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:21,827 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8F8A5910924A47518721068D266FD66B/F7788F2A05C443E5AC4321B2F87870CE/.tez/application_1463537241790_0001_wd
2016-05-18 02:07:21,851 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537241790_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:21,851 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:21,851 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:21,853 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:21,853 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:21,853 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:21,854 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:21,854 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:21,856 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:21,857 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:21,857 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:21,857 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:21,857 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:21,858 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:21,858 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:21,858 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:21,858 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:21,859 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:21,859 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:21,859 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:21,862 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:21,864 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:21,864 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:21,864 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8F8A5910924A47518721068D266FD66B/F7788F2A05C443E5AC4321B2F87870CE/.tez/application_1463537241790_0001_wd/localmode-log-dir
2016-05-18 02:07:21,865 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8F8A5910924A47518721068D266FD66B/F7788F2A05C443E5AC4321B2F87870CE/.tez/application_1463537241790_0001_wd/localmode-log-dir/history.txt.appattempt_1463537241790_0001_000000, maxErrors=10
2016-05-18 02:07:21,865 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8F8A5910924A47518721068D266FD66B/F7788F2A05C443E5AC4321B2F87870CE/.tez/application_1463537241790_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:21,866 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537241790_0001_000000, appSubmitTime=1463537241850, launchTime=1463537241850
2016-05-18 02:07:21,869 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:21,875 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:38250
2016-05-18 02:07:21,875 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537241790
2016-05-18 02:07:21,878 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537241790_0001_000000, startTime=1463537241878
2016-05-18 02:07:21,878 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:21,926 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:21,926 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:21,926 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...SULTS.0_'field2'_'field2', applicationId=application_1463537241790_0001, dagName=[8F8A5910924A47518721068D266FD66B/F7788F2A05C443E5AC4321B2F87870CE] every.every/(1/1) ...SULTS.0_'field2'_'field2'
2016-05-18 02:07:21,944 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [8F8A5910924A47518721068D266FD66B/F7788F2A05C443E5AC4321B2F87870CE] every.every/(1/1) ...SULTS.0_'field2'_'field2'
2016-05-18 02:07:21,944 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:21,945 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537241790_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8F8A5910924A47518721068D266FD66B/F7788F2A05C443E5AC4321B2F87870CE/.tez/application_1463537241790_0001_wd/localmode-log-dir/dag_1463537241790_0001_1.dot
2016-05-18 02:07:21,946 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8F8A5910924A47518721068D266FD66B/F7788F2A05C443E5AC4321B2F87870CE/.tez/application_1463537241790_0001_wd/localmode-log-dir/dag_1463537241790_0001_1-tez-dag.pb.txt
2016-05-18 02:07:21,950 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537241790_0001_1
2016-05-18 02:07:21,950 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:21,950 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [8F8A5910924A47518721068D266FD66B/F7788F2A05C443E5AC4321B2F87870CE] every.every/(1/1) ...SULTS.0_'field2'_'field2'
2016-05-18 02:07:21 Running Dag: dag_1463537241790_0001_1
2016-05-18 02:07:21,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241790_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537241790_0001_1, submitTime=1463537241944
2016-05-18 02:07:21,967 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8F8A5910924A47518721068D266FD66B/F7788F2A05C443E5AC4321B2F87870CE/.tez/application_1463537241790_0001/tez-conf.pb" } size: 40947 timestamp: 1463537241000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:21,967 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3]
2016-05-18 02:07:21,967 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3]
2016-05-18 02:07:21,967 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:21,968 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:21,969 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241790_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537241790_0001_1, initTime=1463537241968
2016-05-18 02:07:21,969 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537241790_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:21,969 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:21,970 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241790_0001_1][Event:DAG_STARTED]: dagID=dag_1463537241790_0001_1, startTime=1463537241969
2016-05-18 02:07:21,970 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537241790_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:21,970 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: ADFE4632261A44CD83A7A7389C8903A3 : {6DF9D3DC01BB474DAA4D095863F62CC1={InputName=6DF9D3DC01BB474DAA4D095863F62CC1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:21,970 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 6DF9D3DC01BB474DAA4D095863F62CC1, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:21,970 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3]
2016-05-18 02:07:21,970 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3]
2016-05-18 02:07:21,970 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3]
2016-05-18 02:07:21,970 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3]
2016-05-18 02:07:21,971 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:21,972 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...SULTS.0_'field2'_'field2', applicationId=application_1463537241790_0001, dagName=[8F8A5910924A47518721068D266FD66B/F7788F2A05C443E5AC4321B2F87870CE] every.every/(1/1) ...SULTS.0_'field2'_'field2'
2016-05-18 02:07:21,973 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 6DF9D3DC01BB474DAA4D095863F62CC1 on vertex vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3]
2016-05-18 02:07:21,976 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 6DF9D3DC01BB474DAA4D095863F62CC1 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:21,978 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:21,978 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:21,979 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:21,979 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:21,979 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 6DF9D3DC01BB474DAA4D095863F62CC1 on vertex vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3]
2016-05-18 02:07:21,982 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:21,982 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {6DF9D3DC01BB474DAA4D095863F62CC1=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:21,983 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3] parallelism set to 1
2016-05-18 02:07:21,983 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:21,983 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3], numAdditionalOutputs=1
2016-05-18 02:07:21,983 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=2539111149A94D669B3D9EE99127970A, vertex=vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:21,985 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for ADFE4632261A44CD83A7A7389C8903A3:2539111149A94D669B3D9EE99127970A using oldmapred API
2016-05-18 02:07:21,985 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=2539111149A94D669B3D9EE99127970A, vertexName=ADFE4632261A44CD83A7A7389C8903A3, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:21,986 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241790_0001_1][Event:VERTEX_INITIALIZED]: vertexName=ADFE4632261A44CD83A7A7389C8903A3, vertexId=vertex_1463537241790_0001_1_00, initRequestedTime=1463537241970, initedTime=1463537241983, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:21,986 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3] with distanceFromRoot: 0
2016-05-18 02:07:21,986 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:21,987 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241790_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537241790_0001_1_00, startRequestedTime=1463537241971, startedTime=1463537241987
2016-05-18 02:07:21,987 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241790_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537241790_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:21,987 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:21,988 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in ADFE4632261A44CD83A7A7389C8903A3
2016-05-18 02:07:21,988 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3]
2016-05-18 02:07:21,988 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241790_0001_1][Event:TASK_STARTED]: vertexName=ADFE4632261A44CD83A7A7389C8903A3, taskId=task_1463537241790_0001_1_00_000000, scheduledTime=1463537241988, launchTime=1463537241988
2016-05-18 02:07:21,989 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3]
2016-05-18 02:07:21,990 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:21,991 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537241790_0001_00_000001, launchTime=1463537241991
2016-05-18 02:07:21,992 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537241790_0001_00_000001
2016-05-18 02:07:21,993 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537241790_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537241790_0001_1_00_000000_0
2016-05-18 02:07:21,993 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537241790_0001_1_00_000000_0] started. Is using containerId: [container_1463537241790_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:21,994 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241790_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=ADFE4632261A44CD83A7A7389C8903A3, taskAttemptId=attempt_1463537241790_0001_1_00_000000_0, startTime=1463537241993, containerId=container_1463537241790_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:21 Starting to run new task attempt: attempt_1463537241790_0001_1_00_000000_0
2016-05-18 02:07:21,994 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537241790_0001_1_00_000000_0
2016-05-18 02:07:21,994 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:21,994 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:21,995 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8F8A5910924A47518721068D266FD66B/F7788F2A05C443E5AC4321B2F87870CE] every.every/(1/1) ...SULTS.0_'field2'_'field2', VertexName: ADFE4632261A44CD83A7A7389C8903A3, VertexParallelism: 1, TaskAttemptID:attempt_1463537241790_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6DF9D3DC01BB474DAA4D095863F62CC1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=2539111149A94D669B3D9EE99127970A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:21,995 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:21,995 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537241790_0001_1_00_000000_0
2016-05-18 02:07:21,996 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537241790_0001, with dag id: dag_1463537241790_0001_1
2016-05-18 02:07:21,999 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537241790_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:22,000 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537241790_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:22,001 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:22,003 INFO input.MRInput (MRInput.java:initialize(440)) - 6DF9D3DC01BB474DAA4D095863F62CC1 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:22,003 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 6DF9D3DC01BB474DAA4D095863F62CC1
2016-05-18 02:07:22,010 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: ADFE4632261A44CD83A7A7389C8903A3, ordinal: 0
2016-05-18 02:07:22,010 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ADFE4632261A44CD83A7A7389C8903A3, mem on start (mb), free: 160, total: 386, max: 672
2016-05-18 02:07:22,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:22,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:22,011 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:22,014 INFO output.MROutput (MROutput.java:initialize(421)) - 2539111149A94D669B3D9EE99127970A: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:22,014 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:22,014 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[6DF9D3DC01BB474DAA4D095863F62CC1:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [2539111149A94D669B3D9EE99127970A:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:22,015 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:22,015 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:22,015 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:22,015 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[6DF9D3DC01BB474DAA4D095863F62CC1:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [2539111149A94D669B3D9EE99127970A:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:22,015 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:22,015 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 6DF9D3DC01BB474DAA4D095863F62CC1 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:22,015 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 6DF9D3DC01BB474DAA4D095863F62CC1
2016-05-18 02:07:22,015 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:22,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:22,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:22,016 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537241790_0001_1_00_000000_0
2016-05-18 02:07:22,020 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:22,021 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 6DF9D3DC01BB474DAA4D095863F62CC1
2016-05-18 02:07:22,021 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_'field2'"], id: 2539111149A94D669B3D9EE99127970A
2016-05-18 02:07:22,021 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_'field2'"]
2016-05-18 02:07:22,022 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:22,022 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:22,022 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 6DF9D3DC01BB474DAA4D095863F62CC1 initialized RecordReader from event
2016-05-18 02:07:22,023 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: ADFE4632261A44CD83A7A7389C8903A3, all 1 inputs ready in: 00:00:00.001
2016-05-18 02:07:22,026 INFO output.MROutput (MROutput.java:close(526)) - 2539111149A94D669B3D9EE99127970A closed
2016-05-18 02:07:22,027 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537241790_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:22,028 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537241790_0001_1_00_000000_0
2016-05-18 02:07:22,028 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ADFE4632261A44CD83A7A7389C8903A3, mem on close (mb), free: 158, total: 386, max: 672
2016-05-18 02:07:22,028 INFO output.MROutput (MROutput.java:close(526)) - 2539111149A94D669B3D9EE99127970A closed
2016-05-18 02:07:22,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:22 Completed running task attempt: attempt_1463537241790_0001_1_00_000000_0
2016-05-18 02:07:22,029 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537241790_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:22,029 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537241790_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:22,030 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537241790_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537242023, Process_Duration=5, Process_End_Time=1463537242028, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:22,031 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537241790_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:22,031 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:22,031 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241790_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=ADFE4632261A44CD83A7A7389C8903A3, taskAttemptId=attempt_1463537241790_0001_1_00_000000_0, creationTime=1463537241989, allocationTime=1463537241990, startTime=1463537241993, finishTime=1463537242031, timeTaken=38, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:22,031 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537241790_0001_1_00_000000_0
2016-05-18 02:07:22,032 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241790_0001_1][Event:TASK_FINISHED]: vertexName=ADFE4632261A44CD83A7A7389C8903A3, taskId=task_1463537241790_0001_1_00_000000, startTime=1463537241993, finishTime=1463537242032, timeTaken=39, status=SUCCEEDED, successfulAttemptID=attempt_1463537241790_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537242023, Process_Duration=5, Process_End_Time=1463537242028, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:22,031 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:22,032 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:22,033 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:22,033 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3]
2016-05-18 02:07:22,034 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241790_0001_1][Event:VERTEX_FINISHED]: vertexName=ADFE4632261A44CD83A7A7389C8903A3, vertexId=vertex_1463537241790_0001_1_00, initRequestedTime=1463537241970, initedTime=1463537241983, startRequestedTime=1463537241971, startedTime=1463537241987, finishTime=1463537242033, timeTaken=46, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537242023, Process_Duration=5, Process_End_Time=1463537242028, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537241993, firstTasksToStart=[ task_1463537241790_0001_1_00_000000 ], lastTaskFinishTime=1463537242031, lastTasksToFinish=[ task_1463537241790_0001_1_00_000000 ], minTaskDuration=38, maxTaskDuration=38, avgTaskDuration=38.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537241790_0001_1_00_000000 ], longestDurationTasks=[ task_1463537241790_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:22,035 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:22,035 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241790_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537241790_0001_00_000001, stoppedTime=1463537242035, exitStatus=0
2016-05-18 02:07:22,035 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:22,035 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:22,035 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537241790_0001_1
2016-05-18 02:07:22,036 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241790_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537241790_0001_1
2016-05-18 02:07:22,036 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 2539111149A94D669B3D9EE99127970A for vertex: vertex_1463537241790_0001_1_00 [ADFE4632261A44CD83A7A7389C8903A3], outputName: 2539111149A94D669B3D9EE99127970A
2016-05-18 02:07:22,036 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537241790_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:22,036 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537241790_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:22,037 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537241790_0001_00_000001
2016-05-18 02:07:22,037 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537241790_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:22,037 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537241790_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:22,037 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537241790_0001_00_000001, Exiting
2016-05-18 02:07:22,038 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537241790_0001_00_000001
2016-05-18 02:07:22,038 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537241790_0001_00_000001
2016-05-18 02:07:22,038 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537241790_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:22,039 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537241790_0001_00_000001 completed successfully
2016-05-18 02:07:22,046 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:2539111149A94D669B3D9EE99127970A of vertex/vertexGroup:ADFE4632261A44CD83A7A7389C8903A3 isVertexGroupOutput:false
2016-05-18 02:07:22,046 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:22,047 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537241790_0001_1, queueSize=0
2016-05-18 02:07:22,048 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537241790_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537241790_0001_1, startTime=1463537241969, finishTime=1463537242046, timeTaken=77, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=140, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537242023, Process_Duration=5, Process_End_Time=1463537242028, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:22,048 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537241790_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:22,048 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537241790_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:22 Completed Dag: dag_1463537241790_0001_1
2016-05-18 02:07:22,048 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537241790_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:22,049 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537241790_0001_1_post
2016-05-18 02:07:22,049 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:22,049 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:22,049 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:22,049 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[8F8A5910924A47518721068D266FD66B/F7788F2A05C443E5AC4321B2F87870CE] every.every/(1/1) ...SULTS.0_'field2'_'field2', with id=dag_1463537241790_0001_1
2016-05-18 02:07:22,049 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[8F8A5910924A47518721068D266FD66B/F7788F2A05C443E5AC4321B2F87870CE] every.every/(1/1) ...SULTS.0_'field2'_'field2', with id=dag_1463537241790_0001_1
2016-05-18 02:07:22,056 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...SULTS.0_'field2'_'field2', applicationId=application_1463537241790_0001
2016-05-18 02:07:22,058 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:22,058 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:22,058 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:22,059 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:22,059 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:22,059 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:22,059 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:22,060 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:22,060 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:22,060 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:22,060 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:22,060 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:22,061 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:22,061 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:22,065 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8F8A5910924A47518721068D266FD66B/F7788F2A05C443E5AC4321B2F87870CE/.tez/application_1463537241790_0001
2016-05-18 02:07:22,065 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_'field2'/_temporary
2016-05-18 02:07:22,065 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.309
2016-05-18 02:07:22,140 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_'field2'_'field2'
2016-05-18 02:07:22,177 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:22,178 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:22,217 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-18 02:07:22,227 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-05-18 02:07:22,227 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:22,228 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:22,228 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:22,252 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:22,253 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_RESULTS
2016-05-18 02:07:22,267 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:22,267 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:22,267 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_RESULTS"]
2016-05-18 02:07:22,267 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:22,268 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:22,268 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:22,269 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ESULTS.0_'field2'_RESULTS
2016-05-18 02:07:22,270 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:22,275 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 02:07:22,275 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 02:07:22,285 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:22,289 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:22,290 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:22,290 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:22,300 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/72D02C8E69714DBAB9295CB50DD67D9E/D5202A8890DC45C1BDF0A16E71B6BDD7/.tez/application_1463537242290_0001 doesn't exist and is created
2016-05-18 02:07:22,319 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:22,319 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:22,320 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/72D02C8E69714DBAB9295CB50DD67D9E/D5202A8890DC45C1BDF0A16E71B6BDD7/.tez/application_1463537242290_0001_wd
2016-05-18 02:07:22,336 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537242290_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:22,336 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:22,337 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:22,338 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:22,338 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:22,338 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:22,338 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:22,339 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:22,341 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:22,341 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:22,341 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:22,341 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:22,341 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:22,341 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:22,342 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:22,342 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:22,342 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:22,342 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:22,342 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:22,343 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:22,344 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:22,345 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:22,345 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:22,346 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/72D02C8E69714DBAB9295CB50DD67D9E/D5202A8890DC45C1BDF0A16E71B6BDD7/.tez/application_1463537242290_0001_wd/localmode-log-dir
2016-05-18 02:07:22,346 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/72D02C8E69714DBAB9295CB50DD67D9E/D5202A8890DC45C1BDF0A16E71B6BDD7/.tez/application_1463537242290_0001_wd/localmode-log-dir/history.txt.appattempt_1463537242290_0001_000000, maxErrors=10
2016-05-18 02:07:22,346 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/72D02C8E69714DBAB9295CB50DD67D9E/D5202A8890DC45C1BDF0A16E71B6BDD7/.tez/application_1463537242290_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:22,347 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537242290_0001_000000, appSubmitTime=1463537242335, launchTime=1463537242335
2016-05-18 02:07:22,352 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:22,356 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:59909
2016-05-18 02:07:22,357 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537242290
2016-05-18 02:07:22,358 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537242290_0001_000000, startTime=1463537242358
2016-05-18 02:07:22,358 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:22,419 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:22,420 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:22,420 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ESULTS.0_'field2'_RESULTS, applicationId=application_1463537242290_0001, dagName=[72D02C8E69714DBAB9295CB50DD67D9E/D5202A8890DC45C1BDF0A16E71B6BDD7] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS
2016-05-18 02:07:22,442 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [72D02C8E69714DBAB9295CB50DD67D9E/D5202A8890DC45C1BDF0A16E71B6BDD7] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS
2016-05-18 02:07:22,443 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:22,444 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537242290_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/72D02C8E69714DBAB9295CB50DD67D9E/D5202A8890DC45C1BDF0A16E71B6BDD7/.tez/application_1463537242290_0001_wd/localmode-log-dir/dag_1463537242290_0001_1.dot
2016-05-18 02:07:22,445 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/72D02C8E69714DBAB9295CB50DD67D9E/D5202A8890DC45C1BDF0A16E71B6BDD7/.tez/application_1463537242290_0001_wd/localmode-log-dir/dag_1463537242290_0001_1-tez-dag.pb.txt
2016-05-18 02:07:22,457 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537242290_0001_1
2016-05-18 02:07:22,457 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:22,457 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [72D02C8E69714DBAB9295CB50DD67D9E/D5202A8890DC45C1BDF0A16E71B6BDD7] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS
2016-05-18 02:07:22 Running Dag: dag_1463537242290_0001_1
2016-05-18 02:07:22,473 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242290_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537242290_0001_1, submitTime=1463537242443
2016-05-18 02:07:22,480 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/72D02C8E69714DBAB9295CB50DD67D9E/D5202A8890DC45C1BDF0A16E71B6BDD7/.tez/application_1463537242290_0001/tez-conf.pb" } size: 40947 timestamp: 1463537242000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:22,480 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850]
2016-05-18 02:07:22,480 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850]
2016-05-18 02:07:22,481 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:22,481 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:22,482 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242290_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537242290_0001_1, initTime=1463537242482
2016-05-18 02:07:22,483 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537242290_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:22,485 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:22,486 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242290_0001_1][Event:DAG_STARTED]: dagID=dag_1463537242290_0001_1, startTime=1463537242486
2016-05-18 02:07:22,486 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537242290_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:22,486 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: A5C5D9BE3FBE45D7B032E39651291850 : {643E19DA16F649A7A1E41C3DFCC46FE9={InputName=643E19DA16F649A7A1E41C3DFCC46FE9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:22,486 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 643E19DA16F649A7A1E41C3DFCC46FE9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:22,486 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850]
2016-05-18 02:07:22,487 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ESULTS.0_'field2'_RESULTS, applicationId=application_1463537242290_0001, dagName=[72D02C8E69714DBAB9295CB50DD67D9E/D5202A8890DC45C1BDF0A16E71B6BDD7] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS
2016-05-18 02:07:22,487 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850]
2016-05-18 02:07:22,487 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850]
2016-05-18 02:07:22,487 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850]
2016-05-18 02:07:22,488 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:22,488 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 643E19DA16F649A7A1E41C3DFCC46FE9 on vertex vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850]
2016-05-18 02:07:22,490 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 643E19DA16F649A7A1E41C3DFCC46FE9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:22,492 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:22,492 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:22,492 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:22,492 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:22,493 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 643E19DA16F649A7A1E41C3DFCC46FE9 on vertex vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850]
2016-05-18 02:07:22,494 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:22,494 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {643E19DA16F649A7A1E41C3DFCC46FE9=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:22,495 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850] parallelism set to 1
2016-05-18 02:07:22,495 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:22,495 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850], numAdditionalOutputs=1
2016-05-18 02:07:22,495 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=D7614C99A3A84048B145227AB0C6CFAB, vertex=vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:22,497 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for A5C5D9BE3FBE45D7B032E39651291850:D7614C99A3A84048B145227AB0C6CFAB using oldmapred API
2016-05-18 02:07:22,497 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=D7614C99A3A84048B145227AB0C6CFAB, vertexName=A5C5D9BE3FBE45D7B032E39651291850, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:22,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242290_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A5C5D9BE3FBE45D7B032E39651291850, vertexId=vertex_1463537242290_0001_1_00, initRequestedTime=1463537242486, initedTime=1463537242495, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:22,498 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850] with distanceFromRoot: 0
2016-05-18 02:07:22,499 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:22,499 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242290_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537242290_0001_1_00, startRequestedTime=1463537242488, startedTime=1463537242499
2016-05-18 02:07:22,500 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242290_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537242290_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:22,500 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:22,500 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in A5C5D9BE3FBE45D7B032E39651291850
2016-05-18 02:07:22,501 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850]
2016-05-18 02:07:22,501 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242290_0001_1][Event:TASK_STARTED]: vertexName=A5C5D9BE3FBE45D7B032E39651291850, taskId=task_1463537242290_0001_1_00_000000, scheduledTime=1463537242501, launchTime=1463537242501
2016-05-18 02:07:22,502 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850]
2016-05-18 02:07:22,503 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:22,505 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537242290_0001_00_000001, launchTime=1463537242504
2016-05-18 02:07:22,506 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537242290_0001_00_000001
2016-05-18 02:07:22,506 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537242290_0001, with dag id: dag_1463537242290_0001_1
2016-05-18 02:07:22,507 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537242290_0001_1_00_000000_0] started. Is using containerId: [container_1463537242290_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:22,507 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242290_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A5C5D9BE3FBE45D7B032E39651291850, taskAttemptId=attempt_1463537242290_0001_1_00_000000_0, startTime=1463537242507, containerId=container_1463537242290_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:22,508 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537242290_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537242290_0001_1_00_000000_0
2016-05-18 02:07:22 Starting to run new task attempt: attempt_1463537242290_0001_1_00_000000_0
2016-05-18 02:07:22,509 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537242290_0001_1_00_000000_0
2016-05-18 02:07:22,509 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:22,509 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:22,510 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [72D02C8E69714DBAB9295CB50DD67D9E/D5202A8890DC45C1BDF0A16E71B6BDD7] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS, VertexName: A5C5D9BE3FBE45D7B032E39651291850, VertexParallelism: 1, TaskAttemptID:attempt_1463537242290_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=643E19DA16F649A7A1E41C3DFCC46FE9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D7614C99A3A84048B145227AB0C6CFAB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:22,510 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:22,511 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537242290_0001_1_00_000000_0
2016-05-18 02:07:22,514 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537242290_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:22,514 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537242290_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:22,518 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:22,521 INFO input.MRInput (MRInput.java:initialize(440)) - 643E19DA16F649A7A1E41C3DFCC46FE9 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:22,521 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 643E19DA16F649A7A1E41C3DFCC46FE9
2016-05-18 02:07:22,526 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A5C5D9BE3FBE45D7B032E39651291850, ordinal: 0
2016-05-18 02:07:22,526 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A5C5D9BE3FBE45D7B032E39651291850, mem on start (mb), free: 278, total: 386, max: 672
2016-05-18 02:07:22,527 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:22,527 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:22,527 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:22,530 INFO output.MROutput (MROutput.java:initialize(421)) - D7614C99A3A84048B145227AB0C6CFAB: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:22,530 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:22,530 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[643E19DA16F649A7A1E41C3DFCC46FE9:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [D7614C99A3A84048B145227AB0C6CFAB:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:22,531 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:22,531 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:22,531 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:22,532 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[643E19DA16F649A7A1E41C3DFCC46FE9:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [D7614C99A3A84048B145227AB0C6CFAB:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:22,532 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:22,532 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 643E19DA16F649A7A1E41C3DFCC46FE9 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:22,532 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 643E19DA16F649A7A1E41C3DFCC46FE9
2016-05-18 02:07:22,532 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:22,533 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:22,533 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:22,533 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537242290_0001_1_00_000000_0
2016-05-18 02:07:22,536 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:22,537 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 643E19DA16F649A7A1E41C3DFCC46FE9
2016-05-18 02:07:22,537 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_RESULTS"], id: D7614C99A3A84048B145227AB0C6CFAB
2016-05-18 02:07:22,537 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_RESULTS"]
2016-05-18 02:07:22,538 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:22,538 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:22,538 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 643E19DA16F649A7A1E41C3DFCC46FE9 initialized RecordReader from event
2016-05-18 02:07:22,540 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A5C5D9BE3FBE45D7B032E39651291850, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:22,543 INFO output.MROutput (MROutput.java:close(526)) - D7614C99A3A84048B145227AB0C6CFAB closed
2016-05-18 02:07:22,543 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537242290_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:22,544 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537242290_0001_1_00_000000_0
2016-05-18 02:07:22,544 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A5C5D9BE3FBE45D7B032E39651291850, mem on close (mb), free: 276, total: 386, max: 672
2016-05-18 02:07:22,544 INFO output.MROutput (MROutput.java:close(526)) - D7614C99A3A84048B145227AB0C6CFAB closed
2016-05-18 02:07:22,545 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:22 Completed running task attempt: attempt_1463537242290_0001_1_00_000000_0
2016-05-18 02:07:22,545 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537242290_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:22,546 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537242290_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:22,546 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537242290_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537242540, Process_Duration=4, Process_End_Time=1463537242544, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:22,547 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537242290_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:22,547 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:22,547 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:22,547 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242290_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A5C5D9BE3FBE45D7B032E39651291850, taskAttemptId=attempt_1463537242290_0001_1_00_000000_0, creationTime=1463537242502, allocationTime=1463537242504, startTime=1463537242507, finishTime=1463537242547, timeTaken=40, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:22,548 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537242290_0001_1_00_000000_0
2016-05-18 02:07:22,548 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:22,548 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537242290_0001_00_000001
2016-05-18 02:07:22,548 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242290_0001_1][Event:TASK_FINISHED]: vertexName=A5C5D9BE3FBE45D7B032E39651291850, taskId=task_1463537242290_0001_1_00_000000, startTime=1463537242507, finishTime=1463537242548, timeTaken=41, status=SUCCEEDED, successfulAttemptID=attempt_1463537242290_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537242540, Process_Duration=4, Process_End_Time=1463537242544, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:22,550 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:22,550 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850]
2016-05-18 02:07:22,551 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242290_0001_1][Event:VERTEX_FINISHED]: vertexName=A5C5D9BE3FBE45D7B032E39651291850, vertexId=vertex_1463537242290_0001_1_00, initRequestedTime=1463537242486, initedTime=1463537242495, startRequestedTime=1463537242488, startedTime=1463537242499, finishTime=1463537242550, timeTaken=51, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537242540, Process_Duration=4, Process_End_Time=1463537242544, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537242507, firstTasksToStart=[ task_1463537242290_0001_1_00_000000 ], lastTaskFinishTime=1463537242547, lastTasksToFinish=[ task_1463537242290_0001_1_00_000000 ], minTaskDuration=40, maxTaskDuration=40, avgTaskDuration=40.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537242290_0001_1_00_000000 ], longestDurationTasks=[ task_1463537242290_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:22,551 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:22,552 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242290_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537242290_0001_00_000001, stoppedTime=1463537242552, exitStatus=0
2016-05-18 02:07:22,552 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:22,552 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:22,552 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537242290_0001_1
2016-05-18 02:07:22,552 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242290_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537242290_0001_1
2016-05-18 02:07:22,552 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: D7614C99A3A84048B145227AB0C6CFAB for vertex: vertex_1463537242290_0001_1_00 [A5C5D9BE3FBE45D7B032E39651291850], outputName: D7614C99A3A84048B145227AB0C6CFAB
2016-05-18 02:07:22,552 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537242290_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:22,554 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537242290_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:22,560 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537242290_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:22,561 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537242290_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:22,561 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537242290_0001_00_000001, Exiting
2016-05-18 02:07:22,561 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537242290_0001_00_000001
2016-05-18 02:07:22,561 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537242290_0001_00_000001
2016-05-18 02:07:22,561 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537242290_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:22,562 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537242290_0001_00_000001 completed successfully
2016-05-18 02:07:22,563 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:D7614C99A3A84048B145227AB0C6CFAB of vertex/vertexGroup:A5C5D9BE3FBE45D7B032E39651291850 isVertexGroupOutput:false
2016-05-18 02:07:22,563 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:22,565 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537242290_0001_1, queueSize=0
2016-05-18 02:07:22,565 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242290_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537242290_0001_1, startTime=1463537242485, finishTime=1463537242564, timeTaken=79, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=140, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537242540, Process_Duration=4, Process_End_Time=1463537242544, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:22,565 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537242290_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:22,565 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537242290_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:22 Completed Dag: dag_1463537242290_0001_1
2016-05-18 02:07:22,566 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537242290_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:22,566 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537242290_0001_1_post
2016-05-18 02:07:22,566 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:22,566 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:22,566 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:22,566 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[72D02C8E69714DBAB9295CB50DD67D9E/D5202A8890DC45C1BDF0A16E71B6BDD7] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS, with id=dag_1463537242290_0001_1
2016-05-18 02:07:22,566 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[72D02C8E69714DBAB9295CB50DD67D9E/D5202A8890DC45C1BDF0A16E71B6BDD7] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS, with id=dag_1463537242290_0001_1
2016-05-18 02:07:22,572 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ESULTS.0_'field2'_RESULTS, applicationId=application_1463537242290_0001
2016-05-18 02:07:22,574 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:22,574 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:22,574 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:22,574 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:22,574 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:22,575 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:22,575 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:22,575 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:22,576 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:22,576 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:22,576 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:22,576 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:22,577 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:22,578 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:22,581 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/72D02C8E69714DBAB9295CB50DD67D9E/D5202A8890DC45C1BDF0A16E71B6BDD7/.tez/application_1463537242290_0001
2016-05-18 02:07:22,581 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_RESULTS/_temporary
2016-05-18 02:07:22,582 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.314
2016-05-18 02:07:22,662 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_'field2'_RESULTS
2016-05-18 02:07:22,686 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:22,686 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:22,723 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-18 02:07:22,733 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-05-18 02:07:22,733 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:22,733 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:22,734 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:22,750 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:22,751 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_ALL
2016-05-18 02:07:22,763 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:22,763 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:22,763 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_ALL"]
2016-05-18 02:07:22,763 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:22,763 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:22,763 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:22,766 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...d'_RESULTS.0_'field2'_ALL
2016-05-18 02:07:22,766 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:22,781 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:22,785 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:22,786 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:22,786 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:22,797 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F7A32EA58B3B4A7BBA77393B132339CE/A20F7A430C864E689E8A894009BB8F78/.tez/application_1463537242786_0001 doesn't exist and is created
2016-05-18 02:07:22,818 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:22,818 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:22,818 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F7A32EA58B3B4A7BBA77393B132339CE/A20F7A430C864E689E8A894009BB8F78/.tez/application_1463537242786_0001_wd
2016-05-18 02:07:22,834 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537242786_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:22,835 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:22,835 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:22,837 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:22,837 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:22,838 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:22,838 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:22,841 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:22,843 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:22,843 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:22,843 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:22,844 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:22,844 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:22,844 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:22,844 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:22,844 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:22,845 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:22,845 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:22,845 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:22,845 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:22,847 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:22,848 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:22,848 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:22,849 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F7A32EA58B3B4A7BBA77393B132339CE/A20F7A430C864E689E8A894009BB8F78/.tez/application_1463537242786_0001_wd/localmode-log-dir
2016-05-18 02:07:22,849 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F7A32EA58B3B4A7BBA77393B132339CE/A20F7A430C864E689E8A894009BB8F78/.tez/application_1463537242786_0001_wd/localmode-log-dir/history.txt.appattempt_1463537242786_0001_000000, maxErrors=10
2016-05-18 02:07:22,850 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F7A32EA58B3B4A7BBA77393B132339CE/A20F7A430C864E689E8A894009BB8F78/.tez/application_1463537242786_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:22,850 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537242786_0001_000000, appSubmitTime=1463537242834, launchTime=1463537242834
2016-05-18 02:07:22,857 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:22,861 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:37250
2016-05-18 02:07:22,862 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537242786
2016-05-18 02:07:22,864 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537242786_0001_000000, startTime=1463537242864
2016-05-18 02:07:22,864 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:22,903 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 02:07:22,903 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 02:07:22,918 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:22,918 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:22,918 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...d'_RESULTS.0_'field2'_ALL, applicationId=application_1463537242786_0001, dagName=[F7A32EA58B3B4A7BBA77393B132339CE/A20F7A430C864E689E8A894009BB8F78] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL
2016-05-18 02:07:22,936 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [F7A32EA58B3B4A7BBA77393B132339CE/A20F7A430C864E689E8A894009BB8F78] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL
2016-05-18 02:07:22,936 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:22,937 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537242786_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F7A32EA58B3B4A7BBA77393B132339CE/A20F7A430C864E689E8A894009BB8F78/.tez/application_1463537242786_0001_wd/localmode-log-dir/dag_1463537242786_0001_1.dot
2016-05-18 02:07:22,938 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F7A32EA58B3B4A7BBA77393B132339CE/A20F7A430C864E689E8A894009BB8F78/.tez/application_1463537242786_0001_wd/localmode-log-dir/dag_1463537242786_0001_1-tez-dag.pb.txt
2016-05-18 02:07:22,942 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537242786_0001_1
2016-05-18 02:07:22,942 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:22,943 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [F7A32EA58B3B4A7BBA77393B132339CE/A20F7A430C864E689E8A894009BB8F78] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL
2016-05-18 02:07:22 Running Dag: dag_1463537242786_0001_1
2016-05-18 02:07:22,959 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242786_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537242786_0001_1, submitTime=1463537242936
2016-05-18 02:07:22,960 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F7A32EA58B3B4A7BBA77393B132339CE/A20F7A430C864E689E8A894009BB8F78/.tez/application_1463537242786_0001/tez-conf.pb" } size: 40947 timestamp: 1463537242000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:22,960 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD]
2016-05-18 02:07:22,960 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD]
2016-05-18 02:07:22,961 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:22,961 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:22,962 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242786_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537242786_0001_1, initTime=1463537242961
2016-05-18 02:07:22,962 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537242786_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:22,962 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:22,963 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242786_0001_1][Event:DAG_STARTED]: dagID=dag_1463537242786_0001_1, startTime=1463537242963
2016-05-18 02:07:22,963 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...d'_RESULTS.0_'field2'_ALL, applicationId=application_1463537242786_0001, dagName=[F7A32EA58B3B4A7BBA77393B132339CE/A20F7A430C864E689E8A894009BB8F78] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL
2016-05-18 02:07:22,963 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537242786_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:22,963 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: AF11CBA33B1F493DA0820854B72C95CD : {34D3CCA8DF944FC4B5D83A3904C4F709={InputName=34D3CCA8DF944FC4B5D83A3904C4F709}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:22,963 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 34D3CCA8DF944FC4B5D83A3904C4F709, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:22,963 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD]
2016-05-18 02:07:22,964 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD]
2016-05-18 02:07:22,964 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD]
2016-05-18 02:07:22,964 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD]
2016-05-18 02:07:22,964 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:22,965 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 34D3CCA8DF944FC4B5D83A3904C4F709 on vertex vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD]
2016-05-18 02:07:22,968 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 34D3CCA8DF944FC4B5D83A3904C4F709 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:22,970 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:22,970 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:22,971 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:22,971 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:22,972 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 34D3CCA8DF944FC4B5D83A3904C4F709 on vertex vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD]
2016-05-18 02:07:22,974 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:22,974 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {34D3CCA8DF944FC4B5D83A3904C4F709=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:22,976 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD] parallelism set to 1
2016-05-18 02:07:22,976 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:22,976 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD], numAdditionalOutputs=1
2016-05-18 02:07:22,976 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=AE2AC82FB1FC4702A85B193AD1D6089B, vertex=vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:22,979 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for AF11CBA33B1F493DA0820854B72C95CD:AE2AC82FB1FC4702A85B193AD1D6089B using oldmapred API
2016-05-18 02:07:22,979 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=AE2AC82FB1FC4702A85B193AD1D6089B, vertexName=AF11CBA33B1F493DA0820854B72C95CD, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:22,980 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242786_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AF11CBA33B1F493DA0820854B72C95CD, vertexId=vertex_1463537242786_0001_1_00, initRequestedTime=1463537242963, initedTime=1463537242976, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:22,980 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD] with distanceFromRoot: 0
2016-05-18 02:07:22,981 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:22,981 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242786_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537242786_0001_1_00, startRequestedTime=1463537242964, startedTime=1463537242981
2016-05-18 02:07:22,982 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242786_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537242786_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:22,982 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:22,982 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in AF11CBA33B1F493DA0820854B72C95CD
2016-05-18 02:07:22,983 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD]
2016-05-18 02:07:22,983 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242786_0001_1][Event:TASK_STARTED]: vertexName=AF11CBA33B1F493DA0820854B72C95CD, taskId=task_1463537242786_0001_1_00_000000, scheduledTime=1463537242983, launchTime=1463537242983
2016-05-18 02:07:22,984 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD]
2016-05-18 02:07:22,985 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:22,987 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537242786_0001_00_000001, launchTime=1463537242986
2016-05-18 02:07:22,987 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537242786_0001_00_000001
2016-05-18 02:07:22,987 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537242786_0001, with dag id: dag_1463537242786_0001_1
2016-05-18 02:07:22,989 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537242786_0001_1_00_000000_0] started. Is using containerId: [container_1463537242786_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:22,989 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242786_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AF11CBA33B1F493DA0820854B72C95CD, taskAttemptId=attempt_1463537242786_0001_1_00_000000_0, startTime=1463537242989, containerId=container_1463537242786_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:22,989 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537242786_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537242786_0001_1_00_000000_0
2016-05-18 02:07:22 Starting to run new task attempt: attempt_1463537242786_0001_1_00_000000_0
2016-05-18 02:07:22,990 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537242786_0001_1_00_000000_0
2016-05-18 02:07:22,990 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:22,990 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:22,991 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [F7A32EA58B3B4A7BBA77393B132339CE/A20F7A430C864E689E8A894009BB8F78] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL, VertexName: AF11CBA33B1F493DA0820854B72C95CD, VertexParallelism: 1, TaskAttemptID:attempt_1463537242786_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=34D3CCA8DF944FC4B5D83A3904C4F709, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AE2AC82FB1FC4702A85B193AD1D6089B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:22,991 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:22,991 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537242786_0001_1_00_000000_0
2016-05-18 02:07:22,995 INFO input.MRInput (MRInput.java:initialize(440)) - 34D3CCA8DF944FC4B5D83A3904C4F709 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:22,996 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 34D3CCA8DF944FC4B5D83A3904C4F709
2016-05-18 02:07:22,996 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537242786_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:22,996 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537242786_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:22,996 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:23,004 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AF11CBA33B1F493DA0820854B72C95CD, ordinal: 0
2016-05-18 02:07:23,005 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AF11CBA33B1F493DA0820854B72C95CD, mem on start (mb), free: 187, total: 386, max: 672
2016-05-18 02:07:23,005 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:23,005 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:23,005 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:23,008 INFO output.MROutput (MROutput.java:initialize(421)) - AE2AC82FB1FC4702A85B193AD1D6089B: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:23,008 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:23,008 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[34D3CCA8DF944FC4B5D83A3904C4F709:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [AE2AC82FB1FC4702A85B193AD1D6089B:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:23,008 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:23,009 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:23,009 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:23,009 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[34D3CCA8DF944FC4B5D83A3904C4F709:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [AE2AC82FB1FC4702A85B193AD1D6089B:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:23,009 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:23,009 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 34D3CCA8DF944FC4B5D83A3904C4F709
2016-05-18 02:07:23,009 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 34D3CCA8DF944FC4B5D83A3904C4F709 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:23,009 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:23,009 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:23,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:23,010 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537242786_0001_1_00_000000_0
2016-05-18 02:07:23,012 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:23,012 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 34D3CCA8DF944FC4B5D83A3904C4F709 initialized RecordReader from event
2016-05-18 02:07:23,014 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:23,014 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 34D3CCA8DF944FC4B5D83A3904C4F709
2016-05-18 02:07:23,014 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_ALL"], id: AE2AC82FB1FC4702A85B193AD1D6089B
2016-05-18 02:07:23,015 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_ALL"]
2016-05-18 02:07:23,015 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:23,015 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AF11CBA33B1F493DA0820854B72C95CD, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:23,018 INFO output.MROutput (MROutput.java:close(526)) - AE2AC82FB1FC4702A85B193AD1D6089B closed
2016-05-18 02:07:23,018 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537242786_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:23,019 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537242786_0001_1_00_000000_0
2016-05-18 02:07:23,019 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AF11CBA33B1F493DA0820854B72C95CD, mem on close (mb), free: 185, total: 386, max: 672
2016-05-18 02:07:23,019 INFO output.MROutput (MROutput.java:close(526)) - AE2AC82FB1FC4702A85B193AD1D6089B closed
2016-05-18 02:07:23,019 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:23 Completed running task attempt: attempt_1463537242786_0001_1_00_000000_0
2016-05-18 02:07:23,020 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537242786_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:23,020 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537242786_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:23,021 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537242786_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537243015, Process_Duration=4, Process_End_Time=1463537243019, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:23,021 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537242786_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:23,022 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:23,022 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:23,022 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:23,022 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242786_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AF11CBA33B1F493DA0820854B72C95CD, taskAttemptId=attempt_1463537242786_0001_1_00_000000_0, creationTime=1463537242984, allocationTime=1463537242986, startTime=1463537242989, finishTime=1463537243022, timeTaken=33, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:23,022 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537242786_0001_1_00_000000_0
2016-05-18 02:07:23,023 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242786_0001_1][Event:TASK_FINISHED]: vertexName=AF11CBA33B1F493DA0820854B72C95CD, taskId=task_1463537242786_0001_1_00_000000, startTime=1463537242989, finishTime=1463537243023, timeTaken=34, status=SUCCEEDED, successfulAttemptID=attempt_1463537242786_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537243015, Process_Duration=4, Process_End_Time=1463537243019, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:23,024 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537242786_0001_00_000001
2016-05-18 02:07:23,024 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:23,024 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD]
2016-05-18 02:07:23,025 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242786_0001_1][Event:VERTEX_FINISHED]: vertexName=AF11CBA33B1F493DA0820854B72C95CD, vertexId=vertex_1463537242786_0001_1_00, initRequestedTime=1463537242963, initedTime=1463537242976, startRequestedTime=1463537242964, startedTime=1463537242981, finishTime=1463537243024, timeTaken=43, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537243015, Process_Duration=4, Process_End_Time=1463537243019, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537242989, firstTasksToStart=[ task_1463537242786_0001_1_00_000000 ], lastTaskFinishTime=1463537243022, lastTasksToFinish=[ task_1463537242786_0001_1_00_000000 ], minTaskDuration=33, maxTaskDuration=33, avgTaskDuration=33.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537242786_0001_1_00_000000 ], longestDurationTasks=[ task_1463537242786_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:23,025 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:23,025 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242786_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537242786_0001_00_000001, stoppedTime=1463537243025, exitStatus=0
2016-05-18 02:07:23,026 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:23,026 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:23,026 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537242786_0001_1
2016-05-18 02:07:23,026 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242786_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537242786_0001_1
2016-05-18 02:07:23,026 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537242786_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:23,026 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: AE2AC82FB1FC4702A85B193AD1D6089B for vertex: vertex_1463537242786_0001_1_00 [AF11CBA33B1F493DA0820854B72C95CD], outputName: AE2AC82FB1FC4702A85B193AD1D6089B
2016-05-18 02:07:23,027 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537242786_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:23,034 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537242786_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:23,034 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537242786_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:23,034 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537242786_0001_00_000001, Exiting
2016-05-18 02:07:23,034 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537242786_0001_00_000001
2016-05-18 02:07:23,035 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537242786_0001_00_000001
2016-05-18 02:07:23,036 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:AE2AC82FB1FC4702A85B193AD1D6089B of vertex/vertexGroup:AF11CBA33B1F493DA0820854B72C95CD isVertexGroupOutput:false
2016-05-18 02:07:23,036 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:23,037 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537242786_0001_1, queueSize=0
2016-05-18 02:07:23,038 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537242786_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537242786_0001_1, startTime=1463537242962, finishTime=1463537243036, timeTaken=74, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=130, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537243015, Process_Duration=4, Process_End_Time=1463537243019, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:23,038 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537242786_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:23,038 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537242786_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:23 Completed Dag: dag_1463537242786_0001_1
2016-05-18 02:07:23,038 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537242786_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:23,038 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537242786_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:23,038 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537242786_0001_00_000001 completed successfully
2016-05-18 02:07:23,039 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537242786_0001_1_post
2016-05-18 02:07:23,039 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:23,039 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 1
2016-05-18 02:07:23,039 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:23,039 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[F7A32EA58B3B4A7BBA77393B132339CE/A20F7A430C864E689E8A894009BB8F78] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL, with id=dag_1463537242786_0001_1
2016-05-18 02:07:23,040 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[F7A32EA58B3B4A7BBA77393B132339CE/A20F7A430C864E689E8A894009BB8F78] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL, with id=dag_1463537242786_0001_1
2016-05-18 02:07:23,045 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...d'_RESULTS.0_'field2'_ALL, applicationId=application_1463537242786_0001
2016-05-18 02:07:23,047 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:23,047 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:23,047 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:23,047 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:23,047 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:23,048 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:23,048 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:23,050 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:23,050 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:23,050 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:23,050 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:23,050 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:23,050 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:23,051 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:23,054 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/F7A32EA58B3B4A7BBA77393B132339CE/A20F7A430C864E689E8A894009BB8F78/.tez/application_1463537242786_0001
2016-05-18 02:07:23,054 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_ALL/_temporary
2016-05-18 02:07:23,055 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.291
2016-05-18 02:07:23,130 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_'field2'_ALL
2016-05-18 02:07:23,155 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:23,156 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:23,194 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-18 02:07:23,198 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-18 02:07:23,198 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:23,199 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:23,199 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:23,216 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:23,217 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_-1
2016-05-18 02:07:23,227 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:23,227 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:23,228 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_-1"]
2016-05-18 02:07:23,228 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:23,228 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:23,228 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:23,229 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...eld'_RESULTS.0_UNKNOWN_-1
2016-05-18 02:07:23,229 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:23,252 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:23,257 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:23,258 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:23,258 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:23,267 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/33BC672B82D747B9A9FD4140738856E2/CBB733957C924C4F9D7B2996E6558D8C/.tez/application_1463537243257_0001 doesn't exist and is created
2016-05-18 02:07:23,287 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:23,287 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:23,288 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/33BC672B82D747B9A9FD4140738856E2/CBB733957C924C4F9D7B2996E6558D8C/.tez/application_1463537243257_0001_wd
2016-05-18 02:07:23,303 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537243257_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:23,304 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:23,304 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:23,305 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:23,305 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:23,306 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:23,306 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:23,307 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:23,308 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:23,308 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:23,309 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:23,309 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:23,309 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:23,309 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:23,309 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:23,309 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:23,309 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:23,310 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:23,310 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:23,310 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:23,312 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:23,313 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:23,313 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:23,314 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/33BC672B82D747B9A9FD4140738856E2/CBB733957C924C4F9D7B2996E6558D8C/.tez/application_1463537243257_0001_wd/localmode-log-dir
2016-05-18 02:07:23,314 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/33BC672B82D747B9A9FD4140738856E2/CBB733957C924C4F9D7B2996E6558D8C/.tez/application_1463537243257_0001_wd/localmode-log-dir/history.txt.appattempt_1463537243257_0001_000000, maxErrors=10
2016-05-18 02:07:23,314 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/33BC672B82D747B9A9FD4140738856E2/CBB733957C924C4F9D7B2996E6558D8C/.tez/application_1463537243257_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:23,314 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537243257_0001_000000, appSubmitTime=1463537243303, launchTime=1463537243303
2016-05-18 02:07:23,319 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:23,324 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:60055
2016-05-18 02:07:23,325 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537243257
2016-05-18 02:07:23,326 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537243257_0001_000000, startTime=1463537243326
2016-05-18 02:07:23,326 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:23,388 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:23,388 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:23,388 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.0_UNKNOWN_-1, applicationId=application_1463537243257_0001, dagName=[33BC672B82D747B9A9FD4140738856E2/CBB733957C924C4F9D7B2996E6558D8C] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1
2016-05-18 02:07:23,406 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [33BC672B82D747B9A9FD4140738856E2/CBB733957C924C4F9D7B2996E6558D8C] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1
2016-05-18 02:07:23,407 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:23,407 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537243257_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/33BC672B82D747B9A9FD4140738856E2/CBB733957C924C4F9D7B2996E6558D8C/.tez/application_1463537243257_0001_wd/localmode-log-dir/dag_1463537243257_0001_1.dot
2016-05-18 02:07:23,408 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/33BC672B82D747B9A9FD4140738856E2/CBB733957C924C4F9D7B2996E6558D8C/.tez/application_1463537243257_0001_wd/localmode-log-dir/dag_1463537243257_0001_1-tez-dag.pb.txt
2016-05-18 02:07:23,411 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537243257_0001_1
2016-05-18 02:07:23,411 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:23,412 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [33BC672B82D747B9A9FD4140738856E2/CBB733957C924C4F9D7B2996E6558D8C] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1
2016-05-18 02:07:23 Running Dag: dag_1463537243257_0001_1
2016-05-18 02:07:23,412 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 02:07:23,412 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 02:07:23,428 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243257_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537243257_0001_1, submitTime=1463537243406
2016-05-18 02:07:23,429 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/33BC672B82D747B9A9FD4140738856E2/CBB733957C924C4F9D7B2996E6558D8C/.tez/application_1463537243257_0001/tez-conf.pb" } size: 40947 timestamp: 1463537243000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:23,429 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D]
2016-05-18 02:07:23,429 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D]
2016-05-18 02:07:23,430 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:23,430 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:23,432 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243257_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537243257_0001_1, initTime=1463537243431
2016-05-18 02:07:23,432 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537243257_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:23,432 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:23,432 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243257_0001_1][Event:DAG_STARTED]: dagID=dag_1463537243257_0001_1, startTime=1463537243432
2016-05-18 02:07:23,432 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537243257_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:23,432 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: B72BFA5398AE477180A0A99927E1B35D : {7FD57C6B7FB548D99743A1C6FC193F2E={InputName=7FD57C6B7FB548D99743A1C6FC193F2E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:23,433 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 7FD57C6B7FB548D99743A1C6FC193F2E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:23,433 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D]
2016-05-18 02:07:23,433 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D]
2016-05-18 02:07:23,433 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D]
2016-05-18 02:07:23,433 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D]
2016-05-18 02:07:23,434 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:23,434 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 7FD57C6B7FB548D99743A1C6FC193F2E on vertex vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D]
2016-05-18 02:07:23,435 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.0_UNKNOWN_-1, applicationId=application_1463537243257_0001, dagName=[33BC672B82D747B9A9FD4140738856E2/CBB733957C924C4F9D7B2996E6558D8C] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1
2016-05-18 02:07:23,437 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 7FD57C6B7FB548D99743A1C6FC193F2E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:23,440 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:23,440 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:23,441 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:23,441 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:23,441 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 7FD57C6B7FB548D99743A1C6FC193F2E on vertex vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D]
2016-05-18 02:07:23,444 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:23,445 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {7FD57C6B7FB548D99743A1C6FC193F2E=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:23,447 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D] parallelism set to 1
2016-05-18 02:07:23,447 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:23,448 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D], numAdditionalOutputs=1
2016-05-18 02:07:23,448 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=CE3311EDC0A944E2898F31C3232260EE, vertex=vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:23,450 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for B72BFA5398AE477180A0A99927E1B35D:CE3311EDC0A944E2898F31C3232260EE using oldmapred API
2016-05-18 02:07:23,450 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=CE3311EDC0A944E2898F31C3232260EE, vertexName=B72BFA5398AE477180A0A99927E1B35D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:23,450 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243257_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B72BFA5398AE477180A0A99927E1B35D, vertexId=vertex_1463537243257_0001_1_00, initRequestedTime=1463537243432, initedTime=1463537243448, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:23,450 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D] with distanceFromRoot: 0
2016-05-18 02:07:23,451 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:23,451 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243257_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537243257_0001_1_00, startRequestedTime=1463537243434, startedTime=1463537243451
2016-05-18 02:07:23,451 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243257_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537243257_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:23,451 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:23,452 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in B72BFA5398AE477180A0A99927E1B35D
2016-05-18 02:07:23,453 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D]
2016-05-18 02:07:23,453 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243257_0001_1][Event:TASK_STARTED]: vertexName=B72BFA5398AE477180A0A99927E1B35D, taskId=task_1463537243257_0001_1_00_000000, scheduledTime=1463537243453, launchTime=1463537243453
2016-05-18 02:07:23,453 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D]
2016-05-18 02:07:23,454 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:23,456 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537243257_0001_00_000001, launchTime=1463537243456
2016-05-18 02:07:23,456 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537243257_0001_00_000001
2016-05-18 02:07:23,457 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537243257_0001_1_00_000000_0] started. Is using containerId: [container_1463537243257_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:23,457 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243257_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B72BFA5398AE477180A0A99927E1B35D, taskAttemptId=attempt_1463537243257_0001_1_00_000000_0, startTime=1463537243457, containerId=container_1463537243257_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:23,458 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537243257_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537243257_0001_1_00_000000_0
2016-05-18 02:07:23 Starting to run new task attempt: attempt_1463537243257_0001_1_00_000000_0
2016-05-18 02:07:23,459 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537243257_0001_1_00_000000_0
2016-05-18 02:07:23,459 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537243257_0001, with dag id: dag_1463537243257_0001_1
2016-05-18 02:07:23,459 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:23,460 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:23,460 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [33BC672B82D747B9A9FD4140738856E2/CBB733957C924C4F9D7B2996E6558D8C] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1, VertexName: B72BFA5398AE477180A0A99927E1B35D, VertexParallelism: 1, TaskAttemptID:attempt_1463537243257_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7FD57C6B7FB548D99743A1C6FC193F2E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CE3311EDC0A944E2898F31C3232260EE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:23,460 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:23,462 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537243257_0001_1_00_000000_0
2016-05-18 02:07:23,462 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537243257_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:23,464 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537243257_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:23,467 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:23,468 INFO input.MRInput (MRInput.java:initialize(440)) - 7FD57C6B7FB548D99743A1C6FC193F2E using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:23,469 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 7FD57C6B7FB548D99743A1C6FC193F2E
2016-05-18 02:07:23,475 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B72BFA5398AE477180A0A99927E1B35D, ordinal: 0
2016-05-18 02:07:23,475 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B72BFA5398AE477180A0A99927E1B35D, mem on start (mb), free: 308, total: 387, max: 672
2016-05-18 02:07:23,475 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:23,476 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:23,476 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:23,481 INFO output.MROutput (MROutput.java:initialize(421)) - CE3311EDC0A944E2898F31C3232260EE: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:23,481 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:23,481 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[7FD57C6B7FB548D99743A1C6FC193F2E:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [CE3311EDC0A944E2898F31C3232260EE:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:23,481 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:23,482 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:23,482 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:23,482 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[7FD57C6B7FB548D99743A1C6FC193F2E:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [CE3311EDC0A944E2898F31C3232260EE:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:23,482 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:23,482 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 7FD57C6B7FB548D99743A1C6FC193F2E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:23,482 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:23,482 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 7FD57C6B7FB548D99743A1C6FC193F2E
2016-05-18 02:07:23,482 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:23,483 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:23,483 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537243257_0001_1_00_000000_0
2016-05-18 02:07:23,484 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:23,485 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 7FD57C6B7FB548D99743A1C6FC193F2E initialized RecordReader from event
2016-05-18 02:07:23,486 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:23,486 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 7FD57C6B7FB548D99743A1C6FC193F2E
2016-05-18 02:07:23,486 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_-1"], id: CE3311EDC0A944E2898F31C3232260EE
2016-05-18 02:07:23,487 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_-1"]
2016-05-18 02:07:23,487 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:23,487 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B72BFA5398AE477180A0A99927E1B35D, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:23,491 INFO output.MROutput (MROutput.java:close(526)) - CE3311EDC0A944E2898F31C3232260EE closed
2016-05-18 02:07:23,491 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537243257_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:23,492 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537243257_0001_1_00_000000_0
2016-05-18 02:07:23,492 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B72BFA5398AE477180A0A99927E1B35D, mem on close (mb), free: 306, total: 387, max: 672
2016-05-18 02:07:23,492 INFO output.MROutput (MROutput.java:close(526)) - CE3311EDC0A944E2898F31C3232260EE closed
2016-05-18 02:07:23,492 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:23 Completed running task attempt: attempt_1463537243257_0001_1_00_000000_0
2016-05-18 02:07:23,493 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537243257_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:23,493 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537243257_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:23,493 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537243257_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537243487, Process_Duration=5, Process_End_Time=1463537243492, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:23,494 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537243257_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:23,494 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:23,494 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243257_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B72BFA5398AE477180A0A99927E1B35D, taskAttemptId=attempt_1463537243257_0001_1_00_000000_0, creationTime=1463537243453, allocationTime=1463537243455, startTime=1463537243457, finishTime=1463537243494, timeTaken=37, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:23,495 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537243257_0001_1_00_000000_0
2016-05-18 02:07:23,495 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:23,495 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:23,496 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537243257_0001_00_000001
2016-05-18 02:07:23,496 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243257_0001_1][Event:TASK_FINISHED]: vertexName=B72BFA5398AE477180A0A99927E1B35D, taskId=task_1463537243257_0001_1_00_000000, startTime=1463537243457, finishTime=1463537243495, timeTaken=38, status=SUCCEEDED, successfulAttemptID=attempt_1463537243257_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537243487, Process_Duration=5, Process_End_Time=1463537243492, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:23,496 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:23,497 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D]
2016-05-18 02:07:23,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243257_0001_1][Event:VERTEX_FINISHED]: vertexName=B72BFA5398AE477180A0A99927E1B35D, vertexId=vertex_1463537243257_0001_1_00, initRequestedTime=1463537243432, initedTime=1463537243448, startRequestedTime=1463537243434, startedTime=1463537243451, finishTime=1463537243497, timeTaken=46, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537243487, Process_Duration=5, Process_End_Time=1463537243492, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537243457, firstTasksToStart=[ task_1463537243257_0001_1_00_000000 ], lastTaskFinishTime=1463537243494, lastTasksToFinish=[ task_1463537243257_0001_1_00_000000 ], minTaskDuration=37, maxTaskDuration=37, avgTaskDuration=37.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537243257_0001_1_00_000000 ], longestDurationTasks=[ task_1463537243257_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:23,498 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:23,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243257_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537243257_0001_00_000001, stoppedTime=1463537243498, exitStatus=0
2016-05-18 02:07:23,498 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:23,498 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:23,498 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537243257_0001_1
2016-05-18 02:07:23,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243257_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537243257_0001_1
2016-05-18 02:07:23,498 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537243257_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:23,498 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: CE3311EDC0A944E2898F31C3232260EE for vertex: vertex_1463537243257_0001_1_00 [B72BFA5398AE477180A0A99927E1B35D], outputName: CE3311EDC0A944E2898F31C3232260EE
2016-05-18 02:07:23,499 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537243257_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:23,507 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537243257_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:23,507 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537243257_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:23,507 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537243257_0001_00_000001, Exiting
2016-05-18 02:07:23,507 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537243257_0001_00_000001
2016-05-18 02:07:23,507 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537243257_0001_00_000001
2016-05-18 02:07:23,508 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537243257_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:23,508 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537243257_0001_00_000001 completed successfully
2016-05-18 02:07:23,510 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:CE3311EDC0A944E2898F31C3232260EE of vertex/vertexGroup:B72BFA5398AE477180A0A99927E1B35D isVertexGroupOutput:false
2016-05-18 02:07:23,510 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:23,511 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537243257_0001_1, queueSize=0
2016-05-18 02:07:23,512 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243257_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537243257_0001_1, startTime=1463537243432, finishTime=1463537243510, timeTaken=78, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=150, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537243487, Process_Duration=5, Process_End_Time=1463537243492, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:23,512 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537243257_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:23,512 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537243257_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:23 Completed Dag: dag_1463537243257_0001_1
2016-05-18 02:07:23,512 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537243257_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:23,513 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537243257_0001_1_post
2016-05-18 02:07:23,513 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:23,513 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:23,513 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:23,514 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[33BC672B82D747B9A9FD4140738856E2/CBB733957C924C4F9D7B2996E6558D8C] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1, with id=dag_1463537243257_0001_1
2016-05-18 02:07:23,514 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[33BC672B82D747B9A9FD4140738856E2/CBB733957C924C4F9D7B2996E6558D8C] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1, with id=dag_1463537243257_0001_1
2016-05-18 02:07:23,519 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...eld'_RESULTS.0_UNKNOWN_-1, applicationId=application_1463537243257_0001
2016-05-18 02:07:23,520 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:23,521 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:23,521 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:23,521 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:23,521 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:23,521 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:23,521 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:23,522 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:23,522 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:23,522 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:23,522 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:23,522 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:23,523 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:23,523 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:23,527 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/33BC672B82D747B9A9FD4140738856E2/CBB733957C924C4F9D7B2996E6558D8C/.tez/application_1463537243257_0001
2016-05-18 02:07:23,527 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_-1/_temporary
2016-05-18 02:07:23,528 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.300
2016-05-18 02:07:23,601 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_UNKNOWN_-1
2016-05-18 02:07:23,626 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:23,627 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:23,658 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-18 02:07:23,661 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.006
2016-05-18 02:07:23,662 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 02:07:23,662 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 02:07:23,662 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_UNKNOWN_'field2'
2016-05-18 02:07:23,688 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:23,688 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:23,727 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-18 02:07:23,732 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-18 02:07:23,733 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:23,733 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:23,733 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:23,750 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:23,751 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_RESULTS
2016-05-18 02:07:23,760 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:23,760 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:23,761 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_RESULTS"]
2016-05-18 02:07:23,761 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:23,761 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:23,762 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:23,763 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...RESULTS.0_UNKNOWN_RESULTS
2016-05-18 02:07:23,763 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:23,776 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:23,780 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:23,781 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:23,781 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:23,793 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7EBE29D2831645CA83646378FFEA2134/E49FCDEDFC04459887BCFDE8F5E1CF76/.tez/application_1463537243781_0001 doesn't exist and is created
2016-05-18 02:07:23,814 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:23,815 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:23,815 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7EBE29D2831645CA83646378FFEA2134/E49FCDEDFC04459887BCFDE8F5E1CF76/.tez/application_1463537243781_0001_wd
2016-05-18 02:07:23,832 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537243781_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:23,832 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:23,832 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:23,834 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:23,834 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:23,834 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:23,834 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:23,834 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:23,836 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:23,836 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:23,836 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:23,836 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:23,836 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:23,836 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:23,836 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:23,836 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:23,837 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:23,837 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:23,837 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:23,837 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:23,839 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:23,840 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:23,840 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:23,840 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7EBE29D2831645CA83646378FFEA2134/E49FCDEDFC04459887BCFDE8F5E1CF76/.tez/application_1463537243781_0001_wd/localmode-log-dir
2016-05-18 02:07:23,840 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7EBE29D2831645CA83646378FFEA2134/E49FCDEDFC04459887BCFDE8F5E1CF76/.tez/application_1463537243781_0001_wd/localmode-log-dir/history.txt.appattempt_1463537243781_0001_000000, maxErrors=10
2016-05-18 02:07:23,841 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7EBE29D2831645CA83646378FFEA2134/E49FCDEDFC04459887BCFDE8F5E1CF76/.tez/application_1463537243781_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:23,841 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537243781_0001_000000, appSubmitTime=1463537243831, launchTime=1463537243831
2016-05-18 02:07:23,844 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:23,849 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49904
2016-05-18 02:07:23,849 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537243781
2016-05-18 02:07:23,851 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537243781_0001_000000, startTime=1463537243851
2016-05-18 02:07:23,851 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:23,915 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:23,915 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:23,915 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...RESULTS.0_UNKNOWN_RESULTS, applicationId=application_1463537243781_0001, dagName=[7EBE29D2831645CA83646378FFEA2134/E49FCDEDFC04459887BCFDE8F5E1CF76] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS
2016-05-18 02:07:23,932 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [7EBE29D2831645CA83646378FFEA2134/E49FCDEDFC04459887BCFDE8F5E1CF76] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS
2016-05-18 02:07:23,932 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:23,933 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537243781_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7EBE29D2831645CA83646378FFEA2134/E49FCDEDFC04459887BCFDE8F5E1CF76/.tez/application_1463537243781_0001_wd/localmode-log-dir/dag_1463537243781_0001_1.dot
2016-05-18 02:07:23,933 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7EBE29D2831645CA83646378FFEA2134/E49FCDEDFC04459887BCFDE8F5E1CF76/.tez/application_1463537243781_0001_wd/localmode-log-dir/dag_1463537243781_0001_1-tez-dag.pb.txt
2016-05-18 02:07:23,937 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 02:07:23,937 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 02:07:23,938 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537243781_0001_1
2016-05-18 02:07:23,938 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:23,938 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [7EBE29D2831645CA83646378FFEA2134/E49FCDEDFC04459887BCFDE8F5E1CF76] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS
2016-05-18 02:07:23 Running Dag: dag_1463537243781_0001_1
2016-05-18 02:07:23,954 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243781_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537243781_0001_1, submitTime=1463537243932
2016-05-18 02:07:23,955 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7EBE29D2831645CA83646378FFEA2134/E49FCDEDFC04459887BCFDE8F5E1CF76/.tez/application_1463537243781_0001/tez-conf.pb" } size: 40947 timestamp: 1463537243000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:23,955 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C]
2016-05-18 02:07:23,956 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C]
2016-05-18 02:07:23,956 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:23,956 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:23,956 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243781_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537243781_0001_1, initTime=1463537243956
2016-05-18 02:07:23,957 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537243781_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:23,957 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:23,957 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243781_0001_1][Event:DAG_STARTED]: dagID=dag_1463537243781_0001_1, startTime=1463537243957
2016-05-18 02:07:23,957 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537243781_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:23,957 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: A363DAEEAF7E47C993392EE8401E9C2C : {B8FEADCBB53A41B8A4666005C245154D={InputName=B8FEADCBB53A41B8A4666005C245154D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:23,957 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: B8FEADCBB53A41B8A4666005C245154D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:23,957 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...RESULTS.0_UNKNOWN_RESULTS, applicationId=application_1463537243781_0001, dagName=[7EBE29D2831645CA83646378FFEA2134/E49FCDEDFC04459887BCFDE8F5E1CF76] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS
2016-05-18 02:07:23,958 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C]
2016-05-18 02:07:23,958 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C]
2016-05-18 02:07:23,958 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C]
2016-05-18 02:07:23,958 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C]
2016-05-18 02:07:23,959 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:23,959 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: B8FEADCBB53A41B8A4666005C245154D on vertex vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C]
2016-05-18 02:07:23,960 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input B8FEADCBB53A41B8A4666005C245154D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:23,962 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:23,962 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:23,962 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:23,962 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:23,963 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: B8FEADCBB53A41B8A4666005C245154D on vertex vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C]
2016-05-18 02:07:23,963 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:23,964 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {B8FEADCBB53A41B8A4666005C245154D=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:23,966 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C] parallelism set to 1
2016-05-18 02:07:23,967 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:23,967 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C], numAdditionalOutputs=1
2016-05-18 02:07:23,967 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=049709166A7843FD8D01BE7782F27EC0, vertex=vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:23,969 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for A363DAEEAF7E47C993392EE8401E9C2C:049709166A7843FD8D01BE7782F27EC0 using oldmapred API
2016-05-18 02:07:23,969 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=049709166A7843FD8D01BE7782F27EC0, vertexName=A363DAEEAF7E47C993392EE8401E9C2C, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:23,969 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243781_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A363DAEEAF7E47C993392EE8401E9C2C, vertexId=vertex_1463537243781_0001_1_00, initRequestedTime=1463537243957, initedTime=1463537243967, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:23,970 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C] with distanceFromRoot: 0
2016-05-18 02:07:23,970 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:23,970 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243781_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537243781_0001_1_00, startRequestedTime=1463537243959, startedTime=1463537243970
2016-05-18 02:07:23,970 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243781_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537243781_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:23,970 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:23,971 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in A363DAEEAF7E47C993392EE8401E9C2C
2016-05-18 02:07:23,971 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C]
2016-05-18 02:07:23,972 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243781_0001_1][Event:TASK_STARTED]: vertexName=A363DAEEAF7E47C993392EE8401E9C2C, taskId=task_1463537243781_0001_1_00_000000, scheduledTime=1463537243971, launchTime=1463537243971
2016-05-18 02:07:23,972 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C]
2016-05-18 02:07:23,973 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:23,975 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537243781_0001_00_000001, launchTime=1463537243974
2016-05-18 02:07:23,975 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537243781_0001_00_000001
2016-05-18 02:07:23,976 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537243781_0001_1_00_000000_0] started. Is using containerId: [container_1463537243781_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:23,976 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243781_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A363DAEEAF7E47C993392EE8401E9C2C, taskAttemptId=attempt_1463537243781_0001_1_00_000000_0, startTime=1463537243976, containerId=container_1463537243781_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:23,977 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537243781_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537243781_0001_1_00_000000_0
2016-05-18 02:07:23 Starting to run new task attempt: attempt_1463537243781_0001_1_00_000000_0
2016-05-18 02:07:23,977 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537243781_0001_1_00_000000_0
2016-05-18 02:07:23,977 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:23,978 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:23,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7EBE29D2831645CA83646378FFEA2134/E49FCDEDFC04459887BCFDE8F5E1CF76] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS, VertexName: A363DAEEAF7E47C993392EE8401E9C2C, VertexParallelism: 1, TaskAttemptID:attempt_1463537243781_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B8FEADCBB53A41B8A4666005C245154D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=049709166A7843FD8D01BE7782F27EC0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:23,978 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:23,978 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537243781_0001_1_00_000000_0
2016-05-18 02:07:23,981 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537243781_0001, with dag id: dag_1463537243781_0001_1
2016-05-18 02:07:23,982 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537243781_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:23,984 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537243781_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:23,985 INFO input.MRInput (MRInput.java:initialize(440)) - B8FEADCBB53A41B8A4666005C245154D using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:23,985 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: B8FEADCBB53A41B8A4666005C245154D
2016-05-18 02:07:23,986 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:23,992 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A363DAEEAF7E47C993392EE8401E9C2C, ordinal: 0
2016-05-18 02:07:23,992 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A363DAEEAF7E47C993392EE8401E9C2C, mem on start (mb), free: 198, total: 387, max: 672
2016-05-18 02:07:23,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:23,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:23,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:24,001 INFO output.MROutput (MROutput.java:initialize(421)) - 049709166A7843FD8D01BE7782F27EC0: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:24,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:24,001 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[B8FEADCBB53A41B8A4666005C245154D:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [049709166A7843FD8D01BE7782F27EC0:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:24,001 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:24,002 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:24,002 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:24,002 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[B8FEADCBB53A41B8A4666005C245154D:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [049709166A7843FD8D01BE7782F27EC0:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:24,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:24,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: B8FEADCBB53A41B8A4666005C245154D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:24,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:24,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:24,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: B8FEADCBB53A41B8A4666005C245154D
2016-05-18 02:07:24,003 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:24,003 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537243781_0001_1_00_000000_0
2016-05-18 02:07:24,008 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:24,009 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: B8FEADCBB53A41B8A4666005C245154D
2016-05-18 02:07:24,009 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_RESULTS"], id: 049709166A7843FD8D01BE7782F27EC0
2016-05-18 02:07:24,009 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:24,009 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - B8FEADCBB53A41B8A4666005C245154D initialized RecordReader from event
2016-05-18 02:07:24,009 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_RESULTS"]
2016-05-18 02:07:24,010 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:24,011 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A363DAEEAF7E47C993392EE8401E9C2C, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:24,014 INFO output.MROutput (MROutput.java:close(526)) - 049709166A7843FD8D01BE7782F27EC0 closed
2016-05-18 02:07:24,014 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537243781_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:24,015 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537243781_0001_1_00_000000_0
2016-05-18 02:07:24,015 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A363DAEEAF7E47C993392EE8401E9C2C, mem on close (mb), free: 196, total: 387, max: 672
2016-05-18 02:07:24,015 INFO output.MROutput (MROutput.java:close(526)) - 049709166A7843FD8D01BE7782F27EC0 closed
2016-05-18 02:07:24,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:24 Completed running task attempt: attempt_1463537243781_0001_1_00_000000_0
2016-05-18 02:07:24,017 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537243781_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:24,017 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537243781_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:24,017 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537243781_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537244011, Process_Duration=4, Process_End_Time=1463537244015, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:24,018 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537243781_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:24,019 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:24,019 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:24,019 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:24,019 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243781_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A363DAEEAF7E47C993392EE8401E9C2C, taskAttemptId=attempt_1463537243781_0001_1_00_000000_0, creationTime=1463537243972, allocationTime=1463537243974, startTime=1463537243976, finishTime=1463537244019, timeTaken=43, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:24,020 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537243781_0001_1_00_000000_0
2016-05-18 02:07:24,020 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243781_0001_1][Event:TASK_FINISHED]: vertexName=A363DAEEAF7E47C993392EE8401E9C2C, taskId=task_1463537243781_0001_1_00_000000, startTime=1463537243976, finishTime=1463537244020, timeTaken=44, status=SUCCEEDED, successfulAttemptID=attempt_1463537243781_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537244011, Process_Duration=4, Process_End_Time=1463537244015, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:24,021 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537243781_0001_00_000001
2016-05-18 02:07:24,021 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:24,021 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C]
2016-05-18 02:07:24,022 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243781_0001_1][Event:VERTEX_FINISHED]: vertexName=A363DAEEAF7E47C993392EE8401E9C2C, vertexId=vertex_1463537243781_0001_1_00, initRequestedTime=1463537243957, initedTime=1463537243967, startRequestedTime=1463537243959, startedTime=1463537243970, finishTime=1463537244021, timeTaken=51, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537244011, Process_Duration=4, Process_End_Time=1463537244015, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537243976, firstTasksToStart=[ task_1463537243781_0001_1_00_000000 ], lastTaskFinishTime=1463537244019, lastTasksToFinish=[ task_1463537243781_0001_1_00_000000 ], minTaskDuration=43, maxTaskDuration=43, avgTaskDuration=43.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537243781_0001_1_00_000000 ], longestDurationTasks=[ task_1463537243781_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:24,023 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:24,023 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243781_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537243781_0001_00_000001, stoppedTime=1463537244023, exitStatus=0
2016-05-18 02:07:24,023 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:24,023 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:24,023 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537243781_0001_1
2016-05-18 02:07:24,024 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243781_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537243781_0001_1
2016-05-18 02:07:24,024 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 049709166A7843FD8D01BE7782F27EC0 for vertex: vertex_1463537243781_0001_1_00 [A363DAEEAF7E47C993392EE8401E9C2C], outputName: 049709166A7843FD8D01BE7782F27EC0
2016-05-18 02:07:24,024 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537243781_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:24,024 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537243781_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:24,024 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537243781_0001_00_000001, Exiting
2016-05-18 02:07:24,024 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537243781_0001_00_000001
2016-05-18 02:07:24,024 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537243781_0001_00_000001
2016-05-18 02:07:24,025 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537243781_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:24,026 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537243781_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:24,026 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537243781_0001_00_000001 completed successfully
2016-05-18 02:07:24,026 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(260)) - Ignoring stop request for containerId: container_1463537243781_0001_00_000001
2016-05-18 02:07:24,034 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:049709166A7843FD8D01BE7782F27EC0 of vertex/vertexGroup:A363DAEEAF7E47C993392EE8401E9C2C isVertexGroupOutput:false
2016-05-18 02:07:24,034 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:24,035 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537243781_0001_1, queueSize=0
2016-05-18 02:07:24,036 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537243781_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537243781_0001_1, startTime=1463537243957, finishTime=1463537244034, timeTaken=77, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=150, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537244011, Process_Duration=4, Process_End_Time=1463537244015, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:24,036 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537243781_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:24,036 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537243781_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:24 Completed Dag: dag_1463537243781_0001_1
2016-05-18 02:07:24,036 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537243781_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:24,036 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537243781_0001_1_post
2016-05-18 02:07:24,036 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:24,036 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:24,036 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:24,037 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[7EBE29D2831645CA83646378FFEA2134/E49FCDEDFC04459887BCFDE8F5E1CF76] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS, with id=dag_1463537243781_0001_1
2016-05-18 02:07:24,037 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[7EBE29D2831645CA83646378FFEA2134/E49FCDEDFC04459887BCFDE8F5E1CF76] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS, with id=dag_1463537243781_0001_1
2016-05-18 02:07:24,049 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...RESULTS.0_UNKNOWN_RESULTS, applicationId=application_1463537243781_0001
2016-05-18 02:07:24,051 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:24,051 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:24,051 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:24,051 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:24,052 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:24,052 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:24,052 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:24,054 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:24,054 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:24,054 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:24,055 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:24,055 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:24,055 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:24,056 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:24,059 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7EBE29D2831645CA83646378FFEA2134/E49FCDEDFC04459887BCFDE8F5E1CF76/.tez/application_1463537243781_0001
2016-05-18 02:07:24,059 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_RESULTS/_temporary
2016-05-18 02:07:24,059 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.299
2016-05-18 02:07:24,133 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_UNKNOWN_RESULTS
2016-05-18 02:07:24,158 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:24,159 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:24,197 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-18 02:07:24,202 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-18 02:07:24,203 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:24,203 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:24,203 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:24,219 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:24,220 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_ALL
2016-05-18 02:07:24,230 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:24,230 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:24,231 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_ALL"]
2016-05-18 02:07:24,231 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:24,231 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:24,231 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:24,232 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ld'_RESULTS.0_UNKNOWN_ALL
2016-05-18 02:07:24,232 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:24,246 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:24,250 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:24,250 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:24,251 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:24,260 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/CAF5E308377D48F8BBECD64BCE8E4D59/EA6939FED63643979C327543D7738D34/.tez/application_1463537244250_0001 doesn't exist and is created
2016-05-18 02:07:24,280 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:24,280 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:24,281 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/CAF5E308377D48F8BBECD64BCE8E4D59/EA6939FED63643979C327543D7738D34/.tez/application_1463537244250_0001_wd
2016-05-18 02:07:24,296 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537244250_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:24,296 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:24,296 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:24,298 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:24,299 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:24,299 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:24,299 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:24,299 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:24,301 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:24,301 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:24,301 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:24,302 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:24,302 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:24,302 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:24,302 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:24,302 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:24,302 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:24,302 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:24,302 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:24,303 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:24,304 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:24,305 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:24,305 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:24,305 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/CAF5E308377D48F8BBECD64BCE8E4D59/EA6939FED63643979C327543D7738D34/.tez/application_1463537244250_0001_wd/localmode-log-dir
2016-05-18 02:07:24,305 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/CAF5E308377D48F8BBECD64BCE8E4D59/EA6939FED63643979C327543D7738D34/.tez/application_1463537244250_0001_wd/localmode-log-dir/history.txt.appattempt_1463537244250_0001_000000, maxErrors=10
2016-05-18 02:07:24,306 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/CAF5E308377D48F8BBECD64BCE8E4D59/EA6939FED63643979C327543D7738D34/.tez/application_1463537244250_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:24,306 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537244250_0001_000000, appSubmitTime=1463537244295, launchTime=1463537244295
2016-05-18 02:07:24,311 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:24,315 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:40746
2016-05-18 02:07:24,315 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537244250
2016-05-18 02:07:24,317 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537244250_0001_000000, startTime=1463537244317
2016-05-18 02:07:24,317 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:24,380 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:24,380 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:24,381 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL, applicationId=application_1463537244250_0001, dagName=[CAF5E308377D48F8BBECD64BCE8E4D59/EA6939FED63643979C327543D7738D34] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL
2016-05-18 02:07:24,409 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [CAF5E308377D48F8BBECD64BCE8E4D59/EA6939FED63643979C327543D7738D34] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL
2016-05-18 02:07:24,409 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:24,410 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537244250_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/CAF5E308377D48F8BBECD64BCE8E4D59/EA6939FED63643979C327543D7738D34/.tez/application_1463537244250_0001_wd/localmode-log-dir/dag_1463537244250_0001_1.dot
2016-05-18 02:07:24,410 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/CAF5E308377D48F8BBECD64BCE8E4D59/EA6939FED63643979C327543D7738D34/.tez/application_1463537244250_0001_wd/localmode-log-dir/dag_1463537244250_0001_1-tez-dag.pb.txt
2016-05-18 02:07:24,414 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537244250_0001_1
2016-05-18 02:07:24,414 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:24,414 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [CAF5E308377D48F8BBECD64BCE8E4D59/EA6939FED63643979C327543D7738D34] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL
2016-05-18 02:07:24 Running Dag: dag_1463537244250_0001_1
2016-05-18 02:07:24,429 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244250_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537244250_0001_1, submitTime=1463537244409
2016-05-18 02:07:24,430 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/CAF5E308377D48F8BBECD64BCE8E4D59/EA6939FED63643979C327543D7738D34/.tez/application_1463537244250_0001/tez-conf.pb" } size: 40947 timestamp: 1463537244000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:24,430 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC]
2016-05-18 02:07:24,431 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC]
2016-05-18 02:07:24,431 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:24,431 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:24,432 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244250_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537244250_0001_1, initTime=1463537244432
2016-05-18 02:07:24,433 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537244250_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:24,433 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:24,434 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244250_0001_1][Event:DAG_STARTED]: dagID=dag_1463537244250_0001_1, startTime=1463537244433
2016-05-18 02:07:24,434 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537244250_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:24,434 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: B313050E99E04E20A638F9E979F7DDDC : {250D344D19BA44DB97D364C8228E4965={InputName=250D344D19BA44DB97D364C8228E4965}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:24,434 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 250D344D19BA44DB97D364C8228E4965, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:24,434 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL, applicationId=application_1463537244250_0001, dagName=[CAF5E308377D48F8BBECD64BCE8E4D59/EA6939FED63643979C327543D7738D34] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL
2016-05-18 02:07:24,434 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC]
2016-05-18 02:07:24,435 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC]
2016-05-18 02:07:24,435 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC]
2016-05-18 02:07:24,435 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC]
2016-05-18 02:07:24,435 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:24,436 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 250D344D19BA44DB97D364C8228E4965 on vertex vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC]
2016-05-18 02:07:24,437 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 250D344D19BA44DB97D364C8228E4965 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:24,438 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:24,438 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:24,439 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:24,439 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:24,439 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 250D344D19BA44DB97D364C8228E4965 on vertex vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC]
2016-05-18 02:07:24,440 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:24,440 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {250D344D19BA44DB97D364C8228E4965=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:24,442 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC] parallelism set to 1
2016-05-18 02:07:24,442 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:24,443 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC], numAdditionalOutputs=1
2016-05-18 02:07:24,443 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=D660E9AF48A74723AC6497F6C4D1A48B, vertex=vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:24,444 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for B313050E99E04E20A638F9E979F7DDDC:D660E9AF48A74723AC6497F6C4D1A48B using oldmapred API
2016-05-18 02:07:24,445 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=D660E9AF48A74723AC6497F6C4D1A48B, vertexName=B313050E99E04E20A638F9E979F7DDDC, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:24,445 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244250_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B313050E99E04E20A638F9E979F7DDDC, vertexId=vertex_1463537244250_0001_1_00, initRequestedTime=1463537244434, initedTime=1463537244443, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:24,445 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC] with distanceFromRoot: 0
2016-05-18 02:07:24,445 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:24,446 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244250_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537244250_0001_1_00, startRequestedTime=1463537244435, startedTime=1463537244445
2016-05-18 02:07:24,446 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244250_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537244250_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:24,448 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:24,449 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in B313050E99E04E20A638F9E979F7DDDC
2016-05-18 02:07:24,449 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC]
2016-05-18 02:07:24,451 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244250_0001_1][Event:TASK_STARTED]: vertexName=B313050E99E04E20A638F9E979F7DDDC, taskId=task_1463537244250_0001_1_00_000000, scheduledTime=1463537244451, launchTime=1463537244451
2016-05-18 02:07:24,451 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC]
2016-05-18 02:07:24,452 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:24,454 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537244250_0001_00_000001, launchTime=1463537244454
2016-05-18 02:07:24,457 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537244250_0001, with dag id: dag_1463537244250_0001_1
2016-05-18 02:07:24,459 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537244250_0001_00_000001
2016-05-18 02:07:24,463 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 02:07:24,463 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 02:07:24,465 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537244250_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537244250_0001_1_00_000000_0
2016-05-18 02:07:24,465 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537244250_0001_1_00_000000_0] started. Is using containerId: [container_1463537244250_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:24,465 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244250_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B313050E99E04E20A638F9E979F7DDDC, taskAttemptId=attempt_1463537244250_0001_1_00_000000_0, startTime=1463537244465, containerId=container_1463537244250_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:24 Starting to run new task attempt: attempt_1463537244250_0001_1_00_000000_0
2016-05-18 02:07:24,467 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537244250_0001_1_00_000000_0
2016-05-18 02:07:24,467 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:24,467 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:24,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [CAF5E308377D48F8BBECD64BCE8E4D59/EA6939FED63643979C327543D7738D34] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL, VertexName: B313050E99E04E20A638F9E979F7DDDC, VertexParallelism: 1, TaskAttemptID:attempt_1463537244250_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=250D344D19BA44DB97D364C8228E4965, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D660E9AF48A74723AC6497F6C4D1A48B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:24,467 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:24,468 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537244250_0001_1_00_000000_0
2016-05-18 02:07:24,468 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537244250_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:24,469 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537244250_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:24,472 INFO input.MRInput (MRInput.java:initialize(440)) - 250D344D19BA44DB97D364C8228E4965 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:24,472 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 250D344D19BA44DB97D364C8228E4965
2016-05-18 02:07:24,473 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:24,483 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B313050E99E04E20A638F9E979F7DDDC, ordinal: 0
2016-05-18 02:07:24,483 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B313050E99E04E20A638F9E979F7DDDC, mem on start (mb), free: 315, total: 386, max: 672
2016-05-18 02:07:24,483 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:24,484 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:24,484 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:24,489 INFO output.MROutput (MROutput.java:initialize(421)) - D660E9AF48A74723AC6497F6C4D1A48B: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:24,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:24,490 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[250D344D19BA44DB97D364C8228E4965:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [D660E9AF48A74723AC6497F6C4D1A48B:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:24,490 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:24,490 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:24,490 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:24,490 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[250D344D19BA44DB97D364C8228E4965:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [D660E9AF48A74723AC6497F6C4D1A48B:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:24,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:24,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 250D344D19BA44DB97D364C8228E4965 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:24,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 250D344D19BA44DB97D364C8228E4965
2016-05-18 02:07:24,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:24,492 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:24,492 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:24,492 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537244250_0001_1_00_000000_0
2016-05-18 02:07:24,493 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:24,494 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 250D344D19BA44DB97D364C8228E4965 initialized RecordReader from event
2016-05-18 02:07:24,496 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:24,497 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 250D344D19BA44DB97D364C8228E4965
2016-05-18 02:07:24,497 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_ALL"], id: D660E9AF48A74723AC6497F6C4D1A48B
2016-05-18 02:07:24,498 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_ALL"]
2016-05-18 02:07:24,498 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:24,498 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B313050E99E04E20A638F9E979F7DDDC, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:24,501 INFO output.MROutput (MROutput.java:close(526)) - D660E9AF48A74723AC6497F6C4D1A48B closed
2016-05-18 02:07:24,502 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537244250_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:24,502 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537244250_0001_1_00_000000_0
2016-05-18 02:07:24,502 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B313050E99E04E20A638F9E979F7DDDC, mem on close (mb), free: 313, total: 386, max: 672
2016-05-18 02:07:24,502 INFO output.MROutput (MROutput.java:close(526)) - D660E9AF48A74723AC6497F6C4D1A48B closed
2016-05-18 02:07:24,503 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:24 Completed running task attempt: attempt_1463537244250_0001_1_00_000000_0
2016-05-18 02:07:24,503 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537244250_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:24,503 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537244250_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:24,504 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537244250_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537244498, Process_Duration=4, Process_End_Time=1463537244502, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:24,505 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537244250_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:24,505 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:24,505 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:24,506 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:24,506 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244250_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B313050E99E04E20A638F9E979F7DDDC, taskAttemptId=attempt_1463537244250_0001_1_00_000000_0, creationTime=1463537244451, allocationTime=1463537244453, startTime=1463537244465, finishTime=1463537244506, timeTaken=41, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:24,506 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537244250_0001_1_00_000000_0
2016-05-18 02:07:24,507 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244250_0001_1][Event:TASK_FINISHED]: vertexName=B313050E99E04E20A638F9E979F7DDDC, taskId=task_1463537244250_0001_1_00_000000, startTime=1463537244465, finishTime=1463537244506, timeTaken=41, status=SUCCEEDED, successfulAttemptID=attempt_1463537244250_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537244498, Process_Duration=4, Process_End_Time=1463537244502, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:24,507 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:24,507 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC]
2016-05-18 02:07:24,509 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537244250_0001_00_000001
2016-05-18 02:07:24,509 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244250_0001_1][Event:VERTEX_FINISHED]: vertexName=B313050E99E04E20A638F9E979F7DDDC, vertexId=vertex_1463537244250_0001_1_00, initRequestedTime=1463537244434, initedTime=1463537244443, startRequestedTime=1463537244435, startedTime=1463537244445, finishTime=1463537244507, timeTaken=62, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537244498, Process_Duration=4, Process_End_Time=1463537244502, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537244465, firstTasksToStart=[ task_1463537244250_0001_1_00_000000 ], lastTaskFinishTime=1463537244506, lastTasksToFinish=[ task_1463537244250_0001_1_00_000000 ], minTaskDuration=41, maxTaskDuration=41, avgTaskDuration=41.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537244250_0001_1_00_000000 ], longestDurationTasks=[ task_1463537244250_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:24,510 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:24,510 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244250_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537244250_0001_00_000001, stoppedTime=1463537244510, exitStatus=0
2016-05-18 02:07:24,510 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:24,510 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:24,510 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537244250_0001_1
2016-05-18 02:07:24,511 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244250_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537244250_0001_1
2016-05-18 02:07:24,511 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537244250_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:24,511 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: D660E9AF48A74723AC6497F6C4D1A48B for vertex: vertex_1463537244250_0001_1_00 [B313050E99E04E20A638F9E979F7DDDC], outputName: D660E9AF48A74723AC6497F6C4D1A48B
2016-05-18 02:07:24,511 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537244250_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:24,519 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537244250_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:24,519 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537244250_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:24,519 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537244250_0001_00_000001, Exiting
2016-05-18 02:07:24,519 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537244250_0001_00_000001
2016-05-18 02:07:24,520 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537244250_0001_00_000001
2016-05-18 02:07:24,521 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537244250_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:24,521 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537244250_0001_00_000001 completed successfully
2016-05-18 02:07:24,521 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:D660E9AF48A74723AC6497F6C4D1A48B of vertex/vertexGroup:B313050E99E04E20A638F9E979F7DDDC isVertexGroupOutput:false
2016-05-18 02:07:24,521 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:24,522 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537244250_0001_1, queueSize=0
2016-05-18 02:07:24,523 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244250_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537244250_0001_1, startTime=1463537244433, finishTime=1463537244521, timeTaken=88, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=160, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537244498, Process_Duration=4, Process_End_Time=1463537244502, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:24,523 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537244250_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:24,523 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537244250_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:24 Completed Dag: dag_1463537244250_0001_1
2016-05-18 02:07:24,524 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537244250_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:24,524 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537244250_0001_1_post
2016-05-18 02:07:24,524 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:24,524 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:24,524 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:24,524 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[CAF5E308377D48F8BBECD64BCE8E4D59/EA6939FED63643979C327543D7738D34] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL, with id=dag_1463537244250_0001_1
2016-05-18 02:07:24,524 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[CAF5E308377D48F8BBECD64BCE8E4D59/EA6939FED63643979C327543D7738D34] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL, with id=dag_1463537244250_0001_1
2016-05-18 02:07:24,530 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL, applicationId=application_1463537244250_0001
2016-05-18 02:07:24,531 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:24,531 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:24,531 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:24,531 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:24,532 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:24,532 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:24,533 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:24,533 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:24,534 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:24,534 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:24,534 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:24,534 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:24,534 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:24,535 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:24,538 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/CAF5E308377D48F8BBECD64BCE8E4D59/EA6939FED63643979C327543D7738D34/.tez/application_1463537244250_0001
2016-05-18 02:07:24,539 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_ALL/_temporary
2016-05-18 02:07:24,539 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.308
2016-05-18 02:07:24,620 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_UNKNOWN_ALL
2016-05-18 02:07:24,643 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:24,644 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:24,680 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-18 02:07:24,687 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-18 02:07:24,688 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:24,688 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:24,688 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:24,703 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:24,703 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_-1
2016-05-18 02:07:24,713 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:24,713 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:24,713 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_-1"]
2016-05-18 02:07:24,713 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:24,713 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:24,714 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:24,715 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-18 02:07:24,715 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:24,728 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:24,733 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:24,733 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:24,733 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:24,742 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9E4DA431939E4074B61692094BF8CC03/D753EC08FD084BE5B86658948B113C7E/.tez/application_1463537244733_0001 doesn't exist and is created
2016-05-18 02:07:24,763 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:24,763 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:24,764 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9E4DA431939E4074B61692094BF8CC03/D753EC08FD084BE5B86658948B113C7E/.tez/application_1463537244733_0001_wd
2016-05-18 02:07:24,779 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537244733_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:24,779 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:24,779 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:24,780 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:24,780 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:24,780 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:24,781 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:24,781 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:24,782 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:24,782 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:24,782 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:24,783 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:24,783 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:24,783 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:24,783 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:24,783 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:24,783 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:24,783 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:24,783 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:24,783 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:24,785 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:24,785 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:24,786 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:24,786 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9E4DA431939E4074B61692094BF8CC03/D753EC08FD084BE5B86658948B113C7E/.tez/application_1463537244733_0001_wd/localmode-log-dir
2016-05-18 02:07:24,786 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9E4DA431939E4074B61692094BF8CC03/D753EC08FD084BE5B86658948B113C7E/.tez/application_1463537244733_0001_wd/localmode-log-dir/history.txt.appattempt_1463537244733_0001_000000, maxErrors=10
2016-05-18 02:07:24,786 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9E4DA431939E4074B61692094BF8CC03/D753EC08FD084BE5B86658948B113C7E/.tez/application_1463537244733_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:24,786 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537244733_0001_000000, appSubmitTime=1463537244778, launchTime=1463537244778
2016-05-18 02:07:24,790 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:24,792 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:35275
2016-05-18 02:07:24,792 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537244733
2016-05-18 02:07:24,799 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537244733_0001_000000, startTime=1463537244799
2016-05-18 02:07:24,799 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:24,863 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:24,863 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:24,864 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ield'_RESULTS.0_VALUES_-1, applicationId=application_1463537244733_0001, dagName=[9E4DA431939E4074B61692094BF8CC03/D753EC08FD084BE5B86658948B113C7E] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-18 02:07:24,880 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [9E4DA431939E4074B61692094BF8CC03/D753EC08FD084BE5B86658948B113C7E] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-18 02:07:24,881 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:24,881 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537244733_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9E4DA431939E4074B61692094BF8CC03/D753EC08FD084BE5B86658948B113C7E/.tez/application_1463537244733_0001_wd/localmode-log-dir/dag_1463537244733_0001_1.dot
2016-05-18 02:07:24,882 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9E4DA431939E4074B61692094BF8CC03/D753EC08FD084BE5B86658948B113C7E/.tez/application_1463537244733_0001_wd/localmode-log-dir/dag_1463537244733_0001_1-tez-dag.pb.txt
2016-05-18 02:07:24,885 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537244733_0001_1
2016-05-18 02:07:24,886 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:24,886 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [9E4DA431939E4074B61692094BF8CC03/D753EC08FD084BE5B86658948B113C7E] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-18 02:07:24 Running Dag: dag_1463537244733_0001_1
2016-05-18 02:07:24,900 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244733_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537244733_0001_1, submitTime=1463537244880
2016-05-18 02:07:24,902 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9E4DA431939E4074B61692094BF8CC03/D753EC08FD084BE5B86658948B113C7E/.tez/application_1463537244733_0001/tez-conf.pb" } size: 40947 timestamp: 1463537244000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:24,902 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8]
2016-05-18 02:07:24,902 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8]
2016-05-18 02:07:24,903 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:24,903 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:24,903 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244733_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537244733_0001_1, initTime=1463537244903
2016-05-18 02:07:24,904 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537244733_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:24,904 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:24,904 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244733_0001_1][Event:DAG_STARTED]: dagID=dag_1463537244733_0001_1, startTime=1463537244904
2016-05-18 02:07:24,904 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537244733_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:24,904 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: CCC139E6FF5B4C03BF1080A5FA0749E8 : {893591437F4D499BBA2BE62EFB8BB0C8={InputName=893591437F4D499BBA2BE62EFB8BB0C8}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:24,904 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 893591437F4D499BBA2BE62EFB8BB0C8, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:24,905 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8]
2016-05-18 02:07:24,905 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8]
2016-05-18 02:07:24,905 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8]
2016-05-18 02:07:24,905 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8]
2016-05-18 02:07:24,905 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:24,906 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 893591437F4D499BBA2BE62EFB8BB0C8 on vertex vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8]
2016-05-18 02:07:24,907 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ield'_RESULTS.0_VALUES_-1, applicationId=application_1463537244733_0001, dagName=[9E4DA431939E4074B61692094BF8CC03/D753EC08FD084BE5B86658948B113C7E] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-18 02:07:24,908 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 893591437F4D499BBA2BE62EFB8BB0C8 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:24,909 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:24,910 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:24,910 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:24,910 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:24,911 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 893591437F4D499BBA2BE62EFB8BB0C8 on vertex vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8]
2016-05-18 02:07:24,912 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:24,913 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {893591437F4D499BBA2BE62EFB8BB0C8=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:24,913 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8] parallelism set to 1
2016-05-18 02:07:24,914 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:24,914 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8], numAdditionalOutputs=1
2016-05-18 02:07:24,914 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=0A33A3B3F9A24F948B99B91D14EA216B, vertex=vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:24,916 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for CCC139E6FF5B4C03BF1080A5FA0749E8:0A33A3B3F9A24F948B99B91D14EA216B using oldmapred API
2016-05-18 02:07:24,916 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=0A33A3B3F9A24F948B99B91D14EA216B, vertexName=CCC139E6FF5B4C03BF1080A5FA0749E8, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:24,917 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244733_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CCC139E6FF5B4C03BF1080A5FA0749E8, vertexId=vertex_1463537244733_0001_1_00, initRequestedTime=1463537244904, initedTime=1463537244914, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:24,917 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8] with distanceFromRoot: 0
2016-05-18 02:07:24,917 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:24,917 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244733_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537244733_0001_1_00, startRequestedTime=1463537244905, startedTime=1463537244917
2016-05-18 02:07:24,918 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244733_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537244733_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:24,918 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:24,918 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in CCC139E6FF5B4C03BF1080A5FA0749E8
2016-05-18 02:07:24,918 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8]
2016-05-18 02:07:24,919 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244733_0001_1][Event:TASK_STARTED]: vertexName=CCC139E6FF5B4C03BF1080A5FA0749E8, taskId=task_1463537244733_0001_1_00_000000, scheduledTime=1463537244919, launchTime=1463537244919
2016-05-18 02:07:24,919 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8]
2016-05-18 02:07:24,920 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:24,921 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537244733_0001_00_000001, launchTime=1463537244921
2016-05-18 02:07:24,922 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537244733_0001_00_000001
2016-05-18 02:07:24,923 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537244733_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537244733_0001_1_00_000000_0
2016-05-18 02:07:24,923 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537244733_0001_1_00_000000_0] started. Is using containerId: [container_1463537244733_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:24,923 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244733_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CCC139E6FF5B4C03BF1080A5FA0749E8, taskAttemptId=attempt_1463537244733_0001_1_00_000000_0, startTime=1463537244923, containerId=container_1463537244733_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:24 Starting to run new task attempt: attempt_1463537244733_0001_1_00_000000_0
2016-05-18 02:07:24,923 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537244733_0001_1_00_000000_0
2016-05-18 02:07:24,923 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:24,924 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:24,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [9E4DA431939E4074B61692094BF8CC03/D753EC08FD084BE5B86658948B113C7E] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1, VertexName: CCC139E6FF5B4C03BF1080A5FA0749E8, VertexParallelism: 1, TaskAttemptID:attempt_1463537244733_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=893591437F4D499BBA2BE62EFB8BB0C8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=0A33A3B3F9A24F948B99B91D14EA216B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:24,924 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:24,925 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537244733_0001_1_00_000000_0
2016-05-18 02:07:24,927 INFO input.MRInput (MRInput.java:initialize(440)) - 893591437F4D499BBA2BE62EFB8BB0C8 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:24,928 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 893591437F4D499BBA2BE62EFB8BB0C8
2016-05-18 02:07:24,928 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:24,929 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537244733_0001, with dag id: dag_1463537244733_0001_1
2016-05-18 02:07:24,932 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537244733_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:24,932 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537244733_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:24,939 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CCC139E6FF5B4C03BF1080A5FA0749E8, ordinal: 0
2016-05-18 02:07:24,939 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CCC139E6FF5B4C03BF1080A5FA0749E8, mem on start (mb), free: 225, total: 386, max: 672
2016-05-18 02:07:24,940 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:24,940 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:24,940 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:24,943 INFO output.MROutput (MROutput.java:initialize(421)) - 0A33A3B3F9A24F948B99B91D14EA216B: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:24,944 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:24,944 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[893591437F4D499BBA2BE62EFB8BB0C8:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [0A33A3B3F9A24F948B99B91D14EA216B:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:24,945 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:24,945 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:24,945 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:24,945 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[893591437F4D499BBA2BE62EFB8BB0C8:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [0A33A3B3F9A24F948B99B91D14EA216B:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:24,945 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:24,945 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 893591437F4D499BBA2BE62EFB8BB0C8
2016-05-18 02:07:24,945 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 893591437F4D499BBA2BE62EFB8BB0C8 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:24,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:24,948 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:24,948 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:24,948 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537244733_0001_1_00_000000_0
2016-05-18 02:07:24,950 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:24,950 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 893591437F4D499BBA2BE62EFB8BB0C8 initialized RecordReader from event
2016-05-18 02:07:24,953 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:24,953 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 893591437F4D499BBA2BE62EFB8BB0C8
2016-05-18 02:07:24,954 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_-1"], id: 0A33A3B3F9A24F948B99B91D14EA216B
2016-05-18 02:07:24,954 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_-1"]
2016-05-18 02:07:24,954 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:24,954 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CCC139E6FF5B4C03BF1080A5FA0749E8, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:24,955 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 02:07:24,957 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 02:07:24,960 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 02:07:24,966 INFO output.MROutput (MROutput.java:close(526)) - 0A33A3B3F9A24F948B99B91D14EA216B closed
2016-05-18 02:07:24,967 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537244733_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:24,968 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537244733_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:24,968 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537244733_0001_1_00_000000_0: Counters: 8 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463537244955, Process_Duration=12, Process_End_Time=1463537244967, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-18 02:07:24,968 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(817)) - Joining on EventRouter
2016-05-18 02:07:24,969 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CCC139E6FF5B4C03BF1080A5FA0749E8, mem on close (mb), free: 223, total: 386, max: 672
2016-05-18 02:07:24,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(851)) - Closed processor for vertex=CCC139E6FF5B4C03BF1080A5FA0749E8, index=0, interruptedStatus=false
2016-05-18 02:07:24,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(887)) - Closed input for vertex=CCC139E6FF5B4C03BF1080A5FA0749E8, sourceVertex=893591437F4D499BBA2BE62EFB8BB0C8, interruptedStatus=true
2016-05-18 02:07:24,969 INFO output.MROutput (MROutput.java:close(526)) - 0A33A3B3F9A24F948B99B91D14EA216B closed
2016-05-18 02:07:24,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(911)) - Closed input for vertex=CCC139E6FF5B4C03BF1080A5FA0749E8, sourceVertex=0A33A3B3F9A24F948B99B91D14EA216B, interruptedStatus=true
2016-05-18 02:07:24,971 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537244733_0001_1_00_000000_0 : TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 02:07:24,971 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 02:07:24,971 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537244733_0001_00_000001
2016-05-18 02:07:24,971 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244733_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CCC139E6FF5B4C03BF1080A5FA0749E8, taskAttemptId=attempt_1463537244733_0001_1_00_000000_0, creationTime=1463537244919, allocationTime=1463537244921, startTime=1463537244923, finishTime=1463537244971, timeTaken=48, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task: attempt_1463537244733_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
, counters=Counters: 9, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463537244955, Process_Duration=12, Process_End_Time=1463537244967, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-18 02:07:24,971 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537244733_0001_00_000001
2016-05-18 02:07:24,972 INFO impl.TaskImpl (TaskImpl.java:transition(1170)) - Failing task: task_1463537244733_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-18 02:07:24,972 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244733_0001_1][Event:TASK_FINISHED]: vertexName=CCC139E6FF5B4C03BF1080A5FA0749E8, taskId=task_1463537244733_0001_1_00_000000, startTime=1463537244923, finishTime=1463537244972, timeTaken=49, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463537244733_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
], counters=Counters: 0
2016-05-18 02:07:24,972 INFO impl.VertexImpl (VertexImpl.java:transition(3353)) - Failing vertex: vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8] because task failed: task_1463537244733_0001_1_00_000000
2016-05-18 02:07:24,973 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(2102)) - Killing tasks in vertex: vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8] due to trigger: OWN_TASK_FAILURE
2016-05-18 02:07:24,973 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8], tasks=1, failed=1, killed=0, success=0, completed=1, commits=0, err=OWN_TASK_FAILURE
2016-05-18 02:07:24,973 INFO impl.VertexImpl (VertexImpl.java:finishWithTerminationCause(2084)) - Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
2016-05-18 02:07:24,973 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3087)) - Invoking committer abort for vertex, vertexId=vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8]
2016-05-18 02:07:24,973 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537244733_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, errorMessage='TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation'}
2016-05-18 02:07:24,973 INFO impl.VertexImpl (VertexImpl.java:run(3095)) - Invoking committer abort for output=0A33A3B3F9A24F948B99B91D14EA216B, vertexId=vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8]
2016-05-18 02:07:24,973 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(294)) - Container: container_1463537244733_0001_00_000001 completed but with errors
2016-05-18 02:07:24,976 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244733_0001_1][Event:VERTEX_FINISHED]: vertexName=CCC139E6FF5B4C03BF1080A5FA0749E8, vertexId=vertex_1463537244733_0001_1_00, initRequestedTime=1463537244904, initedTime=1463537244914, startRequestedTime=1463537244905, startedTime=1463537244917, finishTime=1463537244975, timeTaken=58, status=FAILED, diagnostics=Task failed, taskId=task_1463537244733_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463537244733_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]]
Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
Vertex vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8] killed/failed due to:OWN_TASK_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=1463537244923, firstTasksToStart=[ task_1463537244733_0001_1_00_000000 ], lastTaskFinishTime=1463537244971, lastTasksToFinish=[ task_1463537244733_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-18 02:07:24,977 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-18 02:07:24,978 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244733_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537244733_0001_00_000001, stoppedTime=1463537244977, exitStatus=-1000
2016-05-18 02:07:24,978 INFO node.AMNodeImpl (AMNodeImpl.java:transition(298)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463537244733_0001_1_00_000000_0 failed: true container: container_1463537244733_0001_00_000001 numFailedTAs: 0
2016-05-18 02:07:24,979 INFO container.AMContainerImpl (AMContainerImpl.java:transition(993)) - Container container_1463537244733_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
2016-05-18 02:07:24,979 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-18 02:07:24,979 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=VERTEX_FAILURE
2016-05-18 02:07:24,979 INFO impl.DAGImpl (DAGImpl.java:finishWithTerminationCause(1350)) - DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
2016-05-18 02:07:24,980 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537244733_0001_1, queueSize=0
2016-05-18 02:07:24,981 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537244733_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537244733_0001_1, startTime=1463537244904, finishTime=1463537244980, timeTaken=76, status=FAILED, diagnostics=Vertex failed, vertexName=CCC139E6FF5B4C03BF1080A5FA0749E8, vertexId=vertex_1463537244733_0001_1_00, diagnostics=[Task failed, taskId=task_1463537244733_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463537244733_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8] killed/failed due to:OWN_TASK_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0, counters=Counters: 4, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, AM_CPU_MILLISECONDS=140, AM_GC_TIME_MILLIS=0
2016-05-18 02:07:24,981 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3082)) - Ignoring multiple aborts for vertex: vertex_1463537244733_0001_1_00 [CCC139E6FF5B4C03BF1080A5FA0749E8]
2016-05-18 02:07:24,981 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537244733_0001_1 finished with state: FAILED
2016-05-18 02:07:24,982 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537244733_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:24,982 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(260)) - Ignoring stop request for containerId: container_1463537244733_0001_00_000001
2016-05-18 02:07:24 Completed Dag: dag_1463537244733_0001_1
2016-05-18 02:07:24,982 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537244733_0001_1, dagState=FAILED
2016-05-18 02:07:24,982 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537244733_0001_1_post
2016-05-18 02:07:24,982 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:24,982 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 1
2016-05-18 02:07:24,982 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:24,982 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[9E4DA431939E4074B61692094BF8CC03/D753EC08FD084BE5B86658948B113C7E] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1, with id=dag_1463537244733_0001_1
2016-05-18 02:07:24,982 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[9E4DA431939E4074B61692094BF8CC03/D753EC08FD084BE5B86658948B113C7E] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1, with id=dag_1463537244733_0001_1
2016-05-18 02:07:24,986 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 02:07:24,986 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 02:07:24,996 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ield'_RESULTS.0_VALUES_-1, applicationId=application_1463537244733_0001
2016-05-18 02:07:24,997 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:24,997 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:24,997 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:24,998 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:24,998 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:24,999 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:24,999 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:24,999 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:25,000 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:25,000 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:25,000 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:25,000 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:25,001 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:25,002 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:25,011 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9E4DA431939E4074B61692094BF8CC03/D753EC08FD084BE5B86658948B113C7E/.tez/application_1463537244733_0001
2016-05-18 02:07:25,011 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-18 02:07:25,011 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-18 02:07:25,012 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-18 02:07:25,012 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_-1/_temporary
2016-05-18 02:07:25,013 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.299
2016-05-18 02:07:25,013 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_VALUES_-1
2016-05-18 02:07:25,040 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:25,040 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:25,070 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-18 02:07:25,073 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.006
2016-05-18 02:07:25,073 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 02:07:25,073 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 02:07:25,073 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_VALUES_'field2'
2016-05-18 02:07:25,097 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:25,097 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:25,147 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-18 02:07:25,154 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-18 02:07:25,155 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:25,155 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:25,155 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:25,172 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:25,173 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_RESULTS
2016-05-18 02:07:25,183 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:25,184 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:25,184 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_RESULTS"]
2016-05-18 02:07:25,184 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:25,184 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:25,184 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:25,185 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-18 02:07:25,186 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:25,200 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:25,205 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:25,205 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:25,205 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:25,215 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/17C3D46D91D142D4ADFD6181F1E893E2/FE74EF8276FC4F7F9DD360D2C1D93300/.tez/application_1463537245205_0001 doesn't exist and is created
2016-05-18 02:07:25,238 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:25,238 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:25,239 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/17C3D46D91D142D4ADFD6181F1E893E2/FE74EF8276FC4F7F9DD360D2C1D93300/.tez/application_1463537245205_0001_wd
2016-05-18 02:07:25,257 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537245205_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:25,257 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:25,257 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:25,259 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:25,259 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:25,259 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:25,259 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:25,260 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:25,262 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:25,262 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:25,263 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:25,263 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:25,263 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:25,264 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:25,264 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:25,264 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:25,265 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:25,265 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:25,266 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:25,266 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:25,268 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:25,269 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:25,269 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:25,270 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/17C3D46D91D142D4ADFD6181F1E893E2/FE74EF8276FC4F7F9DD360D2C1D93300/.tez/application_1463537245205_0001_wd/localmode-log-dir
2016-05-18 02:07:25,270 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/17C3D46D91D142D4ADFD6181F1E893E2/FE74EF8276FC4F7F9DD360D2C1D93300/.tez/application_1463537245205_0001_wd/localmode-log-dir/history.txt.appattempt_1463537245205_0001_000000, maxErrors=10
2016-05-18 02:07:25,271 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/17C3D46D91D142D4ADFD6181F1E893E2/FE74EF8276FC4F7F9DD360D2C1D93300/.tez/application_1463537245205_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:25,271 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537245205_0001_000000, appSubmitTime=1463537245256, launchTime=1463537245256
2016-05-18 02:07:25,274 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:25,277 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:33783
2016-05-18 02:07:25,277 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537245205
2016-05-18 02:07:25,280 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537245205_0001_000000, startTime=1463537245280
2016-05-18 02:07:25,281 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:25,338 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:25,338 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:25,339 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ..._RESULTS.0_VALUES_RESULTS, applicationId=application_1463537245205_0001, dagName=[17C3D46D91D142D4ADFD6181F1E893E2/FE74EF8276FC4F7F9DD360D2C1D93300] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-18 02:07:25,356 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [17C3D46D91D142D4ADFD6181F1E893E2/FE74EF8276FC4F7F9DD360D2C1D93300] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-18 02:07:25,356 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:25,357 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537245205_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/17C3D46D91D142D4ADFD6181F1E893E2/FE74EF8276FC4F7F9DD360D2C1D93300/.tez/application_1463537245205_0001_wd/localmode-log-dir/dag_1463537245205_0001_1.dot
2016-05-18 02:07:25,358 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/17C3D46D91D142D4ADFD6181F1E893E2/FE74EF8276FC4F7F9DD360D2C1D93300/.tez/application_1463537245205_0001_wd/localmode-log-dir/dag_1463537245205_0001_1-tez-dag.pb.txt
2016-05-18 02:07:25,363 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537245205_0001_1
2016-05-18 02:07:25,363 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:25,363 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [17C3D46D91D142D4ADFD6181F1E893E2/FE74EF8276FC4F7F9DD360D2C1D93300] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-18 02:07:25 Running Dag: dag_1463537245205_0001_1
2016-05-18 02:07:25,379 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245205_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537245205_0001_1, submitTime=1463537245356
2016-05-18 02:07:25,381 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/17C3D46D91D142D4ADFD6181F1E893E2/FE74EF8276FC4F7F9DD360D2C1D93300/.tez/application_1463537245205_0001/tez-conf.pb" } size: 40947 timestamp: 1463537245000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:25,381 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC]
2016-05-18 02:07:25,381 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC]
2016-05-18 02:07:25,381 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:25,381 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:25,381 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245205_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537245205_0001_1, initTime=1463537245381
2016-05-18 02:07:25,381 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537245205_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:25,382 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:25,382 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245205_0001_1][Event:DAG_STARTED]: dagID=dag_1463537245205_0001_1, startTime=1463537245382
2016-05-18 02:07:25,382 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537245205_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:25,382 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: DBDA18251DFE4CBC97297452276CC6AC : {DC9D8DA485824BDABBA9E5104F15CA11={InputName=DC9D8DA485824BDABBA9E5104F15CA11}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:25,382 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: DC9D8DA485824BDABBA9E5104F15CA11, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:25,382 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC]
2016-05-18 02:07:25,383 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC]
2016-05-18 02:07:25,383 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC]
2016-05-18 02:07:25,383 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ..._RESULTS.0_VALUES_RESULTS, applicationId=application_1463537245205_0001, dagName=[17C3D46D91D142D4ADFD6181F1E893E2/FE74EF8276FC4F7F9DD360D2C1D93300] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-18 02:07:25,383 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC]
2016-05-18 02:07:25,384 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:25,384 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: DC9D8DA485824BDABBA9E5104F15CA11 on vertex vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC]
2016-05-18 02:07:25,386 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input DC9D8DA485824BDABBA9E5104F15CA11 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:25,388 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:25,388 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:25,388 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:25,388 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:25,389 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: DC9D8DA485824BDABBA9E5104F15CA11 on vertex vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC]
2016-05-18 02:07:25,391 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:25,391 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {DC9D8DA485824BDABBA9E5104F15CA11=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:25,392 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC] parallelism set to 1
2016-05-18 02:07:25,392 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:25,392 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC], numAdditionalOutputs=1
2016-05-18 02:07:25,393 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=C98D94F906F644C89A50CEAF4C41F12E, vertex=vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:25,395 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for DBDA18251DFE4CBC97297452276CC6AC:C98D94F906F644C89A50CEAF4C41F12E using oldmapred API
2016-05-18 02:07:25,395 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=C98D94F906F644C89A50CEAF4C41F12E, vertexName=DBDA18251DFE4CBC97297452276CC6AC, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:25,396 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245205_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DBDA18251DFE4CBC97297452276CC6AC, vertexId=vertex_1463537245205_0001_1_00, initRequestedTime=1463537245382, initedTime=1463537245392, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:25,396 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC] with distanceFromRoot: 0
2016-05-18 02:07:25,396 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:25,396 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245205_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537245205_0001_1_00, startRequestedTime=1463537245384, startedTime=1463537245396
2016-05-18 02:07:25,396 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245205_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537245205_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:25,396 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:25,397 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in DBDA18251DFE4CBC97297452276CC6AC
2016-05-18 02:07:25,397 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC]
2016-05-18 02:07:25,397 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245205_0001_1][Event:TASK_STARTED]: vertexName=DBDA18251DFE4CBC97297452276CC6AC, taskId=task_1463537245205_0001_1_00_000000, scheduledTime=1463537245397, launchTime=1463537245397
2016-05-18 02:07:25,398 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC]
2016-05-18 02:07:25,398 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:25,400 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537245205_0001_00_000001, launchTime=1463537245400
2016-05-18 02:07:25,401 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537245205_0001_00_000001
2016-05-18 02:07:25,402 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537245205_0001_1_00_000000_0] started. Is using containerId: [container_1463537245205_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:25,402 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245205_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DBDA18251DFE4CBC97297452276CC6AC, taskAttemptId=attempt_1463537245205_0001_1_00_000000_0, startTime=1463537245401, containerId=container_1463537245205_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:25,402 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537245205_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537245205_0001_1_00_000000_0
2016-05-18 02:07:25 Starting to run new task attempt: attempt_1463537245205_0001_1_00_000000_0
2016-05-18 02:07:25,403 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537245205_0001_1_00_000000_0
2016-05-18 02:07:25,403 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:25,403 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537245205_0001, with dag id: dag_1463537245205_0001_1
2016-05-18 02:07:25,403 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:25,404 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 02:07:25,404 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 02:07:25,404 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [17C3D46D91D142D4ADFD6181F1E893E2/FE74EF8276FC4F7F9DD360D2C1D93300] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS, VertexName: DBDA18251DFE4CBC97297452276CC6AC, VertexParallelism: 1, TaskAttemptID:attempt_1463537245205_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DC9D8DA485824BDABBA9E5104F15CA11, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C98D94F906F644C89A50CEAF4C41F12E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:25,405 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:25,405 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537245205_0001_1_00_000000_0
2016-05-18 02:07:25,407 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537245205_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:25,407 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537245205_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:25,408 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:25,409 INFO input.MRInput (MRInput.java:initialize(440)) - DC9D8DA485824BDABBA9E5104F15CA11 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:25,410 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: DC9D8DA485824BDABBA9E5104F15CA11
2016-05-18 02:07:25,416 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DBDA18251DFE4CBC97297452276CC6AC, ordinal: 0
2016-05-18 02:07:25,417 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DBDA18251DFE4CBC97297452276CC6AC, mem on start (mb), free: 137, total: 386, max: 672
2016-05-18 02:07:25,417 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:25,417 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:25,417 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:25,423 INFO output.MROutput (MROutput.java:initialize(421)) - C98D94F906F644C89A50CEAF4C41F12E: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:25,423 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:25,423 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[DC9D8DA485824BDABBA9E5104F15CA11:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [C98D94F906F644C89A50CEAF4C41F12E:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:25,423 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:25,423 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:25,424 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:25,424 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[DC9D8DA485824BDABBA9E5104F15CA11:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [C98D94F906F644C89A50CEAF4C41F12E:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:25,424 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:25,424 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: DC9D8DA485824BDABBA9E5104F15CA11 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:25,424 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:25,424 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:25,425 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: DC9D8DA485824BDABBA9E5104F15CA11
2016-05-18 02:07:25,425 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:25,425 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537245205_0001_1_00_000000_0
2016-05-18 02:07:25,428 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:25,428 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - DC9D8DA485824BDABBA9E5104F15CA11 initialized RecordReader from event
2016-05-18 02:07:25,429 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:25,430 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: DC9D8DA485824BDABBA9E5104F15CA11
2016-05-18 02:07:25,430 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_RESULTS"], id: C98D94F906F644C89A50CEAF4C41F12E
2016-05-18 02:07:25,430 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_RESULTS"]
2016-05-18 02:07:25,430 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:25,431 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DBDA18251DFE4CBC97297452276CC6AC, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:25,431 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 02:07:25,435 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 02:07:25,439 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 02:07:25,443 INFO output.MROutput (MROutput.java:close(526)) - C98D94F906F644C89A50CEAF4C41F12E closed
2016-05-18 02:07:25,444 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537245205_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:25,445 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537245205_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:25,445 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537245205_0001_1_00_000000_0: Counters: 8 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463537245431, Process_Duration=13, Process_End_Time=1463537245444, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-18 02:07:25,445 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(817)) - Joining on EventRouter
2016-05-18 02:07:25,446 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DBDA18251DFE4CBC97297452276CC6AC, mem on close (mb), free: 135, total: 386, max: 672
2016-05-18 02:07:25,446 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(851)) - Closed processor for vertex=DBDA18251DFE4CBC97297452276CC6AC, index=0, interruptedStatus=false
2016-05-18 02:07:25,446 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(887)) - Closed input for vertex=DBDA18251DFE4CBC97297452276CC6AC, sourceVertex=DC9D8DA485824BDABBA9E5104F15CA11, interruptedStatus=true
2016-05-18 02:07:25,446 INFO output.MROutput (MROutput.java:close(526)) - C98D94F906F644C89A50CEAF4C41F12E closed
2016-05-18 02:07:25,447 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(911)) - Closed input for vertex=DBDA18251DFE4CBC97297452276CC6AC, sourceVertex=C98D94F906F644C89A50CEAF4C41F12E, interruptedStatus=true
2016-05-18 02:07:25,448 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537245205_0001_1_00_000000_0 : TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 02:07:25,448 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 02:07:25,448 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537245205_0001_00_000001
2016-05-18 02:07:25,448 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537245205_0001_00_000001
2016-05-18 02:07:25,448 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245205_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DBDA18251DFE4CBC97297452276CC6AC, taskAttemptId=attempt_1463537245205_0001_1_00_000000_0, creationTime=1463537245397, allocationTime=1463537245399, startTime=1463537245401, finishTime=1463537245448, timeTaken=47, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task: attempt_1463537245205_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
, counters=Counters: 9, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463537245431, Process_Duration=13, Process_End_Time=1463537245444, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-18 02:07:25,449 INFO impl.TaskImpl (TaskImpl.java:transition(1170)) - Failing task: task_1463537245205_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-18 02:07:25,449 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245205_0001_1][Event:TASK_FINISHED]: vertexName=DBDA18251DFE4CBC97297452276CC6AC, taskId=task_1463537245205_0001_1_00_000000, startTime=1463537245401, finishTime=1463537245449, timeTaken=48, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463537245205_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
], counters=Counters: 0
2016-05-18 02:07:25,450 INFO impl.VertexImpl (VertexImpl.java:transition(3353)) - Failing vertex: vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC] because task failed: task_1463537245205_0001_1_00_000000
2016-05-18 02:07:25,450 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(2102)) - Killing tasks in vertex: vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC] due to trigger: OWN_TASK_FAILURE
2016-05-18 02:07:25,450 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537245205_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, errorMessage='TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation'}
2016-05-18 02:07:25,450 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC], tasks=1, failed=1, killed=0, success=0, completed=1, commits=0, err=OWN_TASK_FAILURE
2016-05-18 02:07:25,450 INFO impl.VertexImpl (VertexImpl.java:finishWithTerminationCause(2084)) - Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
2016-05-18 02:07:25,450 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3087)) - Invoking committer abort for vertex, vertexId=vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC]
2016-05-18 02:07:25,450 INFO impl.VertexImpl (VertexImpl.java:run(3095)) - Invoking committer abort for output=C98D94F906F644C89A50CEAF4C41F12E, vertexId=vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC]
2016-05-18 02:07:25,453 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(294)) - Container: container_1463537245205_0001_00_000001 completed but with errors
2016-05-18 02:07:25,455 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245205_0001_1][Event:VERTEX_FINISHED]: vertexName=DBDA18251DFE4CBC97297452276CC6AC, vertexId=vertex_1463537245205_0001_1_00, initRequestedTime=1463537245382, initedTime=1463537245392, startRequestedTime=1463537245384, startedTime=1463537245396, finishTime=1463537245454, timeTaken=58, status=FAILED, diagnostics=Task failed, taskId=task_1463537245205_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463537245205_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]]
Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
Vertex vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC] killed/failed due to:OWN_TASK_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=1463537245401, firstTasksToStart=[ task_1463537245205_0001_1_00_000000 ], lastTaskFinishTime=1463537245448, lastTasksToFinish=[ task_1463537245205_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-18 02:07:25,455 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-18 02:07:25,455 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245205_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537245205_0001_00_000001, stoppedTime=1463537245455, exitStatus=-1000
2016-05-18 02:07:25,456 INFO node.AMNodeImpl (AMNodeImpl.java:transition(298)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463537245205_0001_1_00_000000_0 failed: true container: container_1463537245205_0001_00_000001 numFailedTAs: 0
2016-05-18 02:07:25,456 INFO container.AMContainerImpl (AMContainerImpl.java:transition(993)) - Container container_1463537245205_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
2016-05-18 02:07:25,456 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-18 02:07:25,456 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=VERTEX_FAILURE
2016-05-18 02:07:25,456 INFO impl.DAGImpl (DAGImpl.java:finishWithTerminationCause(1350)) - DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
2016-05-18 02:07:25,459 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537245205_0001_1, queueSize=0
2016-05-18 02:07:25,459 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245205_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537245205_0001_1, startTime=1463537245382, finishTime=1463537245459, timeTaken=77, status=FAILED, diagnostics=Vertex failed, vertexName=DBDA18251DFE4CBC97297452276CC6AC, vertexId=vertex_1463537245205_0001_1_00, diagnostics=[Task failed, taskId=task_1463537245205_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463537245205_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC] killed/failed due to:OWN_TASK_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0, counters=Counters: 4, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, AM_CPU_MILLISECONDS=130, AM_GC_TIME_MILLIS=0
2016-05-18 02:07:25,460 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3082)) - Ignoring multiple aborts for vertex: vertex_1463537245205_0001_1_00 [DBDA18251DFE4CBC97297452276CC6AC]
2016-05-18 02:07:25,460 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537245205_0001_1 finished with state: FAILED
2016-05-18 02:07:25,460 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537245205_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:25,460 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(260)) - Ignoring stop request for containerId: container_1463537245205_0001_00_000001
2016-05-18 02:07:25 Completed Dag: dag_1463537245205_0001_1
2016-05-18 02:07:25,461 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537245205_0001_1, dagState=FAILED
2016-05-18 02:07:25,461 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537245205_0001_1_post
2016-05-18 02:07:25,462 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:25,462 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 1
2016-05-18 02:07:25,462 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:25,462 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[17C3D46D91D142D4ADFD6181F1E893E2/FE74EF8276FC4F7F9DD360D2C1D93300] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS, with id=dag_1463537245205_0001_1
2016-05-18 02:07:25,462 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[17C3D46D91D142D4ADFD6181F1E893E2/FE74EF8276FC4F7F9DD360D2C1D93300] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS, with id=dag_1463537245205_0001_1
2016-05-18 02:07:25,467 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ..._RESULTS.0_VALUES_RESULTS, applicationId=application_1463537245205_0001
2016-05-18 02:07:25,469 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:25,469 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:25,469 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:25,469 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:25,469 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:25,470 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:25,470 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:25,470 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:25,470 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:25,471 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:25,471 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:25,471 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:25,471 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:25,472 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:25,475 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/17C3D46D91D142D4ADFD6181F1E893E2/FE74EF8276FC4F7F9DD360D2C1D93300/.tez/application_1463537245205_0001
2016-05-18 02:07:25,476 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-18 02:07:25,476 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-18 02:07:25,477 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-18 02:07:25,477 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_RESULTS/_temporary
2016-05-18 02:07:25,477 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.294
2016-05-18 02:07:25,477 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_VALUES_RESULTS
2016-05-18 02:07:25,514 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:25,515 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:25,564 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-18 02:07:25,571 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-18 02:07:25,571 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:25,571 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:25,572 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:25,587 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:25,587 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_ALL
2016-05-18 02:07:25,597 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:25,597 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:25,597 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_ALL"]
2016-05-18 02:07:25,598 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:25,598 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:25,598 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:25,599 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-18 02:07:25,599 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:25,615 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:25,619 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:25,619 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:25,620 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:25,628 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/4286FE623B2747DE8EB4A0403AD6E18C/ECCC3F9144D946A5ACEAFB39B60B44BD/.tez/application_1463537245619_0001 doesn't exist and is created
2016-05-18 02:07:25,649 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:25,649 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:25,650 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/4286FE623B2747DE8EB4A0403AD6E18C/ECCC3F9144D946A5ACEAFB39B60B44BD/.tez/application_1463537245619_0001_wd
2016-05-18 02:07:25,665 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537245619_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:25,665 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:25,665 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:25,667 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:25,667 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:25,667 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:25,667 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:25,667 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:25,669 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:25,669 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:25,669 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:25,669 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:25,670 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:25,670 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:25,670 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:25,670 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:25,670 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:25,670 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:25,670 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:25,670 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:25,672 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:25,673 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:25,673 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:25,673 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/4286FE623B2747DE8EB4A0403AD6E18C/ECCC3F9144D946A5ACEAFB39B60B44BD/.tez/application_1463537245619_0001_wd/localmode-log-dir
2016-05-18 02:07:25,673 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/4286FE623B2747DE8EB4A0403AD6E18C/ECCC3F9144D946A5ACEAFB39B60B44BD/.tez/application_1463537245619_0001_wd/localmode-log-dir/history.txt.appattempt_1463537245619_0001_000000, maxErrors=10
2016-05-18 02:07:25,674 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/4286FE623B2747DE8EB4A0403AD6E18C/ECCC3F9144D946A5ACEAFB39B60B44BD/.tez/application_1463537245619_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:25,674 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537245619_0001_000000, appSubmitTime=1463537245664, launchTime=1463537245664
2016-05-18 02:07:25,677 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:25,681 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:45602
2016-05-18 02:07:25,682 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537245619
2016-05-18 02:07:25,684 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537245619_0001_000000, startTime=1463537245684
2016-05-18 02:07:25,684 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:25,749 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:25,749 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:25,750 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.0_VALUES_ALL, applicationId=application_1463537245619_0001, dagName=[4286FE623B2747DE8EB4A0403AD6E18C/ECCC3F9144D946A5ACEAFB39B60B44BD] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-18 02:07:25,765 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [4286FE623B2747DE8EB4A0403AD6E18C/ECCC3F9144D946A5ACEAFB39B60B44BD] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-18 02:07:25,766 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:25,766 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537245619_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/4286FE623B2747DE8EB4A0403AD6E18C/ECCC3F9144D946A5ACEAFB39B60B44BD/.tez/application_1463537245619_0001_wd/localmode-log-dir/dag_1463537245619_0001_1.dot
2016-05-18 02:07:25,767 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/4286FE623B2747DE8EB4A0403AD6E18C/ECCC3F9144D946A5ACEAFB39B60B44BD/.tez/application_1463537245619_0001_wd/localmode-log-dir/dag_1463537245619_0001_1-tez-dag.pb.txt
2016-05-18 02:07:25,770 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537245619_0001_1
2016-05-18 02:07:25,770 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:25,770 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [4286FE623B2747DE8EB4A0403AD6E18C/ECCC3F9144D946A5ACEAFB39B60B44BD] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-18 02:07:25 Running Dag: dag_1463537245619_0001_1
2016-05-18 02:07:25,785 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245619_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537245619_0001_1, submitTime=1463537245765
2016-05-18 02:07:25,786 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/4286FE623B2747DE8EB4A0403AD6E18C/ECCC3F9144D946A5ACEAFB39B60B44BD/.tez/application_1463537245619_0001/tez-conf.pb" } size: 40947 timestamp: 1463537245000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:25,786 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B]
2016-05-18 02:07:25,786 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B]
2016-05-18 02:07:25,786 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:25,786 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:25,787 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245619_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537245619_0001_1, initTime=1463537245787
2016-05-18 02:07:25,787 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537245619_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:25,787 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:25,788 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245619_0001_1][Event:DAG_STARTED]: dagID=dag_1463537245619_0001_1, startTime=1463537245788
2016-05-18 02:07:25,788 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537245619_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:25,788 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: BF8EDC91651B460ABDCEC1328F2B079B : {457EF9833C19401D8DC6DB9BE6C0C1FD={InputName=457EF9833C19401D8DC6DB9BE6C0C1FD}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:25,788 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 457EF9833C19401D8DC6DB9BE6C0C1FD, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:25,788 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B]
2016-05-18 02:07:25,788 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B]
2016-05-18 02:07:25,789 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B]
2016-05-18 02:07:25,789 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B]
2016-05-18 02:07:25,790 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:25,790 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.0_VALUES_ALL, applicationId=application_1463537245619_0001, dagName=[4286FE623B2747DE8EB4A0403AD6E18C/ECCC3F9144D946A5ACEAFB39B60B44BD] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-18 02:07:25,792 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 457EF9833C19401D8DC6DB9BE6C0C1FD on vertex vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B]
2016-05-18 02:07:25,795 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 457EF9833C19401D8DC6DB9BE6C0C1FD asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:25,797 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:25,797 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:25,798 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:25,798 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:25,799 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 457EF9833C19401D8DC6DB9BE6C0C1FD on vertex vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B]
2016-05-18 02:07:25,801 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:25,802 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {457EF9833C19401D8DC6DB9BE6C0C1FD=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:25,803 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B] parallelism set to 1
2016-05-18 02:07:25,803 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:25,803 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B], numAdditionalOutputs=1
2016-05-18 02:07:25,803 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=C11D820D6B9B4F5FA3BF822A4732B1ED, vertex=vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:25,805 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for BF8EDC91651B460ABDCEC1328F2B079B:C11D820D6B9B4F5FA3BF822A4732B1ED using oldmapred API
2016-05-18 02:07:25,805 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=C11D820D6B9B4F5FA3BF822A4732B1ED, vertexName=BF8EDC91651B460ABDCEC1328F2B079B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:25,806 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245619_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BF8EDC91651B460ABDCEC1328F2B079B, vertexId=vertex_1463537245619_0001_1_00, initRequestedTime=1463537245788, initedTime=1463537245803, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:25,806 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B] with distanceFromRoot: 0
2016-05-18 02:07:25,806 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:25,806 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245619_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537245619_0001_1_00, startRequestedTime=1463537245790, startedTime=1463537245806
2016-05-18 02:07:25,806 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245619_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537245619_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:25,806 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:25,807 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in BF8EDC91651B460ABDCEC1328F2B079B
2016-05-18 02:07:25,807 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B]
2016-05-18 02:07:25,808 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245619_0001_1][Event:TASK_STARTED]: vertexName=BF8EDC91651B460ABDCEC1328F2B079B, taskId=task_1463537245619_0001_1_00_000000, scheduledTime=1463537245808, launchTime=1463537245808
2016-05-18 02:07:25,808 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B]
2016-05-18 02:07:25,809 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:25,811 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537245619_0001_00_000001, launchTime=1463537245810
2016-05-18 02:07:25,812 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537245619_0001_00_000001
2016-05-18 02:07:25,813 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537245619_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537245619_0001_1_00_000000_0
2016-05-18 02:07:25,813 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537245619_0001_1_00_000000_0] started. Is using containerId: [container_1463537245619_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:25,813 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245619_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BF8EDC91651B460ABDCEC1328F2B079B, taskAttemptId=attempt_1463537245619_0001_1_00_000000_0, startTime=1463537245812, containerId=container_1463537245619_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:25 Starting to run new task attempt: attempt_1463537245619_0001_1_00_000000_0
2016-05-18 02:07:25,814 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537245619_0001_1_00_000000_0
2016-05-18 02:07:25,814 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:25,814 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:25,814 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [4286FE623B2747DE8EB4A0403AD6E18C/ECCC3F9144D946A5ACEAFB39B60B44BD] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL, VertexName: BF8EDC91651B460ABDCEC1328F2B079B, VertexParallelism: 1, TaskAttemptID:attempt_1463537245619_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=457EF9833C19401D8DC6DB9BE6C0C1FD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C11D820D6B9B4F5FA3BF822A4732B1ED, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:25,815 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:25,818 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537245619_0001, with dag id: dag_1463537245619_0001_1
2016-05-18 02:07:25,818 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537245619_0001_1_00_000000_0
2016-05-18 02:07:25,820 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537245619_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:25,821 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537245619_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:25,823 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:25,830 INFO input.MRInput (MRInput.java:initialize(440)) - 457EF9833C19401D8DC6DB9BE6C0C1FD using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:25,831 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 457EF9833C19401D8DC6DB9BE6C0C1FD
2016-05-18 02:07:25,834 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BF8EDC91651B460ABDCEC1328F2B079B, ordinal: 0
2016-05-18 02:07:25,834 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BF8EDC91651B460ABDCEC1328F2B079B, mem on start (mb), free: 277, total: 387, max: 672
2016-05-18 02:07:25,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:25,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:25,835 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:25,839 INFO output.MROutput (MROutput.java:initialize(421)) - C11D820D6B9B4F5FA3BF822A4732B1ED: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:25,840 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:25,840 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[C11D820D6B9B4F5FA3BF822A4732B1ED:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [457EF9833C19401D8DC6DB9BE6C0C1FD:INPUT:0:org.apache.tez.mapreduce.input.MRInput]
2016-05-18 02:07:25,840 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:25,840 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:25,840 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:25,841 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[C11D820D6B9B4F5FA3BF822A4732B1ED:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [457EF9833C19401D8DC6DB9BE6C0C1FD:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0]
2016-05-18 02:07:25,841 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:25,841 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 457EF9833C19401D8DC6DB9BE6C0C1FD being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:25,841 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:25,841 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:25,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 457EF9833C19401D8DC6DB9BE6C0C1FD
2016-05-18 02:07:25,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:25,842 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537245619_0001_1_00_000000_0
2016-05-18 02:07:25,846 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:25,846 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 457EF9833C19401D8DC6DB9BE6C0C1FD initialized RecordReader from event
2016-05-18 02:07:25,846 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:25,847 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 457EF9833C19401D8DC6DB9BE6C0C1FD
2016-05-18 02:07:25,849 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_ALL"], id: C11D820D6B9B4F5FA3BF822A4732B1ED
2016-05-18 02:07:25,849 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_ALL"]
2016-05-18 02:07:25,849 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:25,850 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BF8EDC91651B460ABDCEC1328F2B079B, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:25,853 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 02:07:25,854 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 02:07:25,854 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
2016-05-18 02:07:25,855 INFO output.MROutput (MROutput.java:close(526)) - C11D820D6B9B4F5FA3BF822A4732B1ED closed
2016-05-18 02:07:25,856 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537245619_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:25,864 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537245619_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:25,864 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537245619_0001_1_00_000000_0: Counters: 8 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463537245850, Process_Duration=14, Process_End_Time=1463537245864, Read_Duration=2, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-18 02:07:25,864 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(817)) - Joining on EventRouter
2016-05-18 02:07:25,865 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BF8EDC91651B460ABDCEC1328F2B079B, mem on close (mb), free: 275, total: 387, max: 672
2016-05-18 02:07:25,865 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(851)) - Closed processor for vertex=BF8EDC91651B460ABDCEC1328F2B079B, index=0, interruptedStatus=false
2016-05-18 02:07:25,866 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(887)) - Closed input for vertex=BF8EDC91651B460ABDCEC1328F2B079B, sourceVertex=457EF9833C19401D8DC6DB9BE6C0C1FD, interruptedStatus=true
2016-05-18 02:07:25,866 INFO output.MROutput (MROutput.java:close(526)) - C11D820D6B9B4F5FA3BF822A4732B1ED closed
2016-05-18 02:07:25,866 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(911)) - Closed input for vertex=BF8EDC91651B460ABDCEC1328F2B079B, sourceVertex=C11D820D6B9B4F5FA3BF822A4732B1ED, interruptedStatus=true
2016-05-18 02:07:25,868 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537245619_0001_1_00_000000_0 : TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 02:07:25,868 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=TASK_ERROR, error=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, containerShutdownRequested=false}
2016-05-18 02:07:25,868 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537245619_0001_00_000001
2016-05-18 02:07:25,868 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537245619_0001_00_000001
2016-05-18 02:07:25,869 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245619_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BF8EDC91651B460ABDCEC1328F2B079B, taskAttemptId=attempt_1463537245619_0001_1_00_000000_0, creationTime=1463537245808, allocationTime=1463537245810, startTime=1463537245812, finishTime=1463537245869, timeTaken=57, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task: attempt_1463537245619_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
, counters=Counters: 9, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463537245850, Process_Duration=14, Process_End_Time=1463537245864, Read_Duration=2, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-18 02:07:25,869 INFO impl.TaskImpl (TaskImpl.java:transition(1170)) - Failing task: task_1463537245619_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-18 02:07:25,870 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245619_0001_1][Event:TASK_FINISHED]: vertexName=BF8EDC91651B460ABDCEC1328F2B079B, taskId=task_1463537245619_0001_1_00_000000, startTime=1463537245812, finishTime=1463537245870, timeTaken=58, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463537245619_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
], counters=Counters: 0
2016-05-18 02:07:25,870 INFO impl.VertexImpl (VertexImpl.java:transition(3353)) - Failing vertex: vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B] because task failed: task_1463537245619_0001_1_00_000000
2016-05-18 02:07:25,870 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(2102)) - Killing tasks in vertex: vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B] due to trigger: OWN_TASK_FAILURE
2016-05-18 02:07:25,870 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B], tasks=1, failed=1, killed=0, success=0, completed=1, commits=0, err=OWN_TASK_FAILURE
2016-05-18 02:07:25,871 INFO impl.VertexImpl (VertexImpl.java:finishWithTerminationCause(2084)) - Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
2016-05-18 02:07:25,871 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3087)) - Invoking committer abort for vertex, vertexId=vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B]
2016-05-18 02:07:25,871 INFO impl.VertexImpl (VertexImpl.java:run(3095)) - Invoking committer abort for output=C11D820D6B9B4F5FA3BF822A4732B1ED, vertexId=vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B]
2016-05-18 02:07:25,873 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245619_0001_1][Event:VERTEX_FINISHED]: vertexName=BF8EDC91651B460ABDCEC1328F2B079B, vertexId=vertex_1463537245619_0001_1_00, initRequestedTime=1463537245788, initedTime=1463537245803, startRequestedTime=1463537245790, startedTime=1463537245806, finishTime=1463537245872, timeTaken=66, status=FAILED, diagnostics=Task failed, taskId=task_1463537245619_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463537245619_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]]
Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
Vertex vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B] killed/failed due to:OWN_TASK_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=1463537245812, firstTasksToStart=[ task_1463537245619_0001_1_00_000000 ], lastTaskFinishTime=1463537245869, lastTasksToFinish=[ task_1463537245619_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-18 02:07:25,875 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-18 02:07:25,876 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-18 02:07:25,876 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=VERTEX_FAILURE
2016-05-18 02:07:25,876 INFO impl.DAGImpl (DAGImpl.java:finishWithTerminationCause(1350)) - DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
2016-05-18 02:07:25,878 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537245619_0001_1, queueSize=0
2016-05-18 02:07:25,878 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245619_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537245619_0001_1, startTime=1463537245787, finishTime=1463537245878, timeTaken=91, status=FAILED, diagnostics=Vertex failed, vertexName=BF8EDC91651B460ABDCEC1328F2B079B, vertexId=vertex_1463537245619_0001_1_00, diagnostics=[Task failed, taskId=task_1463537245619_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1463537245619_0001_1_00_000000_0:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 24 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B] killed/failed due to:OWN_TASK_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0, counters=Counters: 4, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, AM_CPU_MILLISECONDS=230, AM_GC_TIME_MILLIS=0
2016-05-18 02:07:25,879 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3082)) - Ignoring multiple aborts for vertex: vertex_1463537245619_0001_1_00 [BF8EDC91651B460ABDCEC1328F2B079B]
2016-05-18 02:07:25,879 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537245619_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation, errorMessage='TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation'}
2016-05-18 02:07:25,879 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(294)) - Container: container_1463537245619_0001_00_000001 completed but with errors
2016-05-18 02:07:25,879 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537245619_0001_1 finished with state: FAILED
2016-05-18 02:07:25,880 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537245619_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:25,880 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537245619_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537245619_0001_00_000001, stoppedTime=1463537245880, exitStatus=-1000
2016-05-18 02:07:25,880 INFO node.AMNodeImpl (AMNodeImpl.java:transition(298)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463537245619_0001_1_00_000000_0 failed: true container: container_1463537245619_0001_00_000001 numFailedTAs: 0
2016-05-18 02:07:25 Completed Dag: dag_1463537245619_0001_1
2016-05-18 02:07:25,881 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537245619_0001_1, dagState=FAILED
2016-05-18 02:07:25,881 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537245619_0001_1_post
2016-05-18 02:07:25,881 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:25,881 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 3
2016-05-18 02:07:25,881 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:25,882 INFO container.AMContainerImpl (AMContainerImpl.java:transition(993)) - Container container_1463537245619_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
2016-05-18 02:07:25,882 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[4286FE623B2747DE8EB4A0403AD6E18C/ECCC3F9144D946A5ACEAFB39B60B44BD] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL, with id=dag_1463537245619_0001_1
2016-05-18 02:07:25,882 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(260)) - Ignoring stop request for containerId: container_1463537245619_0001_00_000001
2016-05-18 02:07:25,882 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[4286FE623B2747DE8EB4A0403AD6E18C/ECCC3F9144D946A5ACEAFB39B60B44BD] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL, with id=dag_1463537245619_0001_1
2016-05-18 02:07:25,885 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...eld'_RESULTS.0_VALUES_ALL, applicationId=application_1463537245619_0001
2016-05-18 02:07:25,887 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:25,887 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:25,887 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:25,887 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:25,888 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:25,889 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:25,889 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:25,889 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:25,889 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:25,889 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:25,890 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:25,890 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:25,890 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:25,891 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:25,894 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/4286FE623B2747DE8EB4A0403AD6E18C/ECCC3F9144D946A5ACEAFB39B60B44BD/.tez/application_1463537245619_0001
2016-05-18 02:07:25,894 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-18 02:07:25,894 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-18 02:07:25,895 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-18 02:07:25,895 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_ALL/_temporary
2016-05-18 02:07:25,896 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.298
2016-05-18 02:07:25,896 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_VALUES_ALL
2016-05-18 02:07:25,931 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:25,932 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:25,936 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 02:07:25,936 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 02:07:25,974 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-18 02:07:25,980 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-18 02:07:25,980 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:25,981 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:25,982 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:25,997 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:25,998 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_-1
2016-05-18 02:07:26,008 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:26,008 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:26,008 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_-1"]
2016-05-18 02:07:26,008 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:26,009 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:26,009 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:26,010 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...'field'_RESULTS.0_ARGS_-1
2016-05-18 02:07:26,010 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:26,025 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:26,029 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:26,029 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:26,030 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:26,038 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/719CBD48C9EB4E32BAFDA1D0C4341390/A182D403E8DD4A368067427C832C5AD6/.tez/application_1463537246029_0001 doesn't exist and is created
2016-05-18 02:07:26,059 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:26,059 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:26,059 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/719CBD48C9EB4E32BAFDA1D0C4341390/A182D403E8DD4A368067427C832C5AD6/.tez/application_1463537246029_0001_wd
2016-05-18 02:07:26,075 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537246029_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:26,075 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:26,075 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:26,077 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:26,077 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:26,077 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:26,077 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:26,078 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:26,079 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:26,079 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:26,079 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:26,079 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:26,079 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:26,079 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:26,080 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:26,080 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:26,080 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:26,080 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:26,080 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:26,080 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:26,082 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:26,082 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:26,082 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:26,083 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/719CBD48C9EB4E32BAFDA1D0C4341390/A182D403E8DD4A368067427C832C5AD6/.tez/application_1463537246029_0001_wd/localmode-log-dir
2016-05-18 02:07:26,083 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/719CBD48C9EB4E32BAFDA1D0C4341390/A182D403E8DD4A368067427C832C5AD6/.tez/application_1463537246029_0001_wd/localmode-log-dir/history.txt.appattempt_1463537246029_0001_000000, maxErrors=10
2016-05-18 02:07:26,083 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/719CBD48C9EB4E32BAFDA1D0C4341390/A182D403E8DD4A368067427C832C5AD6/.tez/application_1463537246029_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:26,083 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537246029_0001_000000, appSubmitTime=1463537246074, launchTime=1463537246074
2016-05-18 02:07:26,086 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:26,091 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:45431
2016-05-18 02:07:26,091 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537246029
2016-05-18 02:07:26,093 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537246029_0001_000000, startTime=1463537246093
2016-05-18 02:07:26,093 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:26,159 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:26,159 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:26,159 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...'field'_RESULTS.0_ARGS_-1, applicationId=application_1463537246029_0001, dagName=[719CBD48C9EB4E32BAFDA1D0C4341390/A182D403E8DD4A368067427C832C5AD6] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1
2016-05-18 02:07:26,176 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [719CBD48C9EB4E32BAFDA1D0C4341390/A182D403E8DD4A368067427C832C5AD6] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1
2016-05-18 02:07:26,176 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:26,176 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537246029_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/719CBD48C9EB4E32BAFDA1D0C4341390/A182D403E8DD4A368067427C832C5AD6/.tez/application_1463537246029_0001_wd/localmode-log-dir/dag_1463537246029_0001_1.dot
2016-05-18 02:07:26,177 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/719CBD48C9EB4E32BAFDA1D0C4341390/A182D403E8DD4A368067427C832C5AD6/.tez/application_1463537246029_0001_wd/localmode-log-dir/dag_1463537246029_0001_1-tez-dag.pb.txt
2016-05-18 02:07:26,180 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537246029_0001_1
2016-05-18 02:07:26,180 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:26,180 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [719CBD48C9EB4E32BAFDA1D0C4341390/A182D403E8DD4A368067427C832C5AD6] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1
2016-05-18 02:07:26 Running Dag: dag_1463537246029_0001_1
2016-05-18 02:07:26,195 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246029_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537246029_0001_1, submitTime=1463537246176
2016-05-18 02:07:26,196 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/719CBD48C9EB4E32BAFDA1D0C4341390/A182D403E8DD4A368067427C832C5AD6/.tez/application_1463537246029_0001/tez-conf.pb" } size: 40947 timestamp: 1463537246000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:26,197 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672]
2016-05-18 02:07:26,197 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672]
2016-05-18 02:07:26,197 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:26,197 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:26,199 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246029_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537246029_0001_1, initTime=1463537246199
2016-05-18 02:07:26,199 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537246029_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:26,199 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:26,200 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246029_0001_1][Event:DAG_STARTED]: dagID=dag_1463537246029_0001_1, startTime=1463537246199
2016-05-18 02:07:26,200 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537246029_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:26,200 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: B4A75DA5A5A94AE8B208E398D2481672 : {68F324BD7F684E389FDF0862906B5F4F={InputName=68F324BD7F684E389FDF0862906B5F4F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:26,200 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 68F324BD7F684E389FDF0862906B5F4F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:26,200 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672]
2016-05-18 02:07:26,200 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...'field'_RESULTS.0_ARGS_-1, applicationId=application_1463537246029_0001, dagName=[719CBD48C9EB4E32BAFDA1D0C4341390/A182D403E8DD4A368067427C832C5AD6] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1
2016-05-18 02:07:26,201 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672]
2016-05-18 02:07:26,201 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672]
2016-05-18 02:07:26,201 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672]
2016-05-18 02:07:26,201 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:26,202 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 68F324BD7F684E389FDF0862906B5F4F on vertex vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672]
2016-05-18 02:07:26,204 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 68F324BD7F684E389FDF0862906B5F4F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:26,206 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:26,206 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:26,206 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:26,206 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:26,207 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 68F324BD7F684E389FDF0862906B5F4F on vertex vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672]
2016-05-18 02:07:26,209 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:26,209 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {68F324BD7F684E389FDF0862906B5F4F=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:26,209 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672] parallelism set to 1
2016-05-18 02:07:26,210 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:26,210 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672], numAdditionalOutputs=1
2016-05-18 02:07:26,210 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=EED8C2DC6B604606B2B6064EF407DD77, vertex=vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:26,211 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for B4A75DA5A5A94AE8B208E398D2481672:EED8C2DC6B604606B2B6064EF407DD77 using oldmapred API
2016-05-18 02:07:26,212 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=EED8C2DC6B604606B2B6064EF407DD77, vertexName=B4A75DA5A5A94AE8B208E398D2481672, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:26,212 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246029_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B4A75DA5A5A94AE8B208E398D2481672, vertexId=vertex_1463537246029_0001_1_00, initRequestedTime=1463537246200, initedTime=1463537246210, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:26,212 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672] with distanceFromRoot: 0
2016-05-18 02:07:26,212 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:26,213 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246029_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537246029_0001_1_00, startRequestedTime=1463537246202, startedTime=1463537246213
2016-05-18 02:07:26,213 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246029_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537246029_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:26,213 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:26,213 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in B4A75DA5A5A94AE8B208E398D2481672
2016-05-18 02:07:26,213 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672]
2016-05-18 02:07:26,215 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246029_0001_1][Event:TASK_STARTED]: vertexName=B4A75DA5A5A94AE8B208E398D2481672, taskId=task_1463537246029_0001_1_00_000000, scheduledTime=1463537246215, launchTime=1463537246215
2016-05-18 02:07:26,215 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672]
2016-05-18 02:07:26,216 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:26,218 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537246029_0001_00_000001, launchTime=1463537246217
2016-05-18 02:07:26,218 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537246029_0001_00_000001
2016-05-18 02:07:26,219 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537246029_0001_1_00_000000_0] started. Is using containerId: [container_1463537246029_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:26,219 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246029_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B4A75DA5A5A94AE8B208E398D2481672, taskAttemptId=attempt_1463537246029_0001_1_00_000000_0, startTime=1463537246219, containerId=container_1463537246029_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:26,220 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537246029_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537246029_0001_1_00_000000_0
2016-05-18 02:07:26 Starting to run new task attempt: attempt_1463537246029_0001_1_00_000000_0
2016-05-18 02:07:26,220 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537246029_0001_1_00_000000_0
2016-05-18 02:07:26,220 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:26,220 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:26,221 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [719CBD48C9EB4E32BAFDA1D0C4341390/A182D403E8DD4A368067427C832C5AD6] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1, VertexName: B4A75DA5A5A94AE8B208E398D2481672, VertexParallelism: 1, TaskAttemptID:attempt_1463537246029_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=68F324BD7F684E389FDF0862906B5F4F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EED8C2DC6B604606B2B6064EF407DD77, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:26,221 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:26,221 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537246029_0001_1_00_000000_0
2016-05-18 02:07:26,222 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537246029_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:26,222 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537246029_0001, with dag id: dag_1463537246029_0001_1
2016-05-18 02:07:26,223 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537246029_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:26,225 INFO input.MRInput (MRInput.java:initialize(440)) - 68F324BD7F684E389FDF0862906B5F4F using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:26,225 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 68F324BD7F684E389FDF0862906B5F4F
2016-05-18 02:07:26,226 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:26,234 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B4A75DA5A5A94AE8B208E398D2481672, ordinal: 0
2016-05-18 02:07:26,234 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B4A75DA5A5A94AE8B208E398D2481672, mem on start (mb), free: 201, total: 387, max: 672
2016-05-18 02:07:26,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:26,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:26,235 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:26,237 INFO output.MROutput (MROutput.java:initialize(421)) - EED8C2DC6B604606B2B6064EF407DD77: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:26,237 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:26,237 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[68F324BD7F684E389FDF0862906B5F4F:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [EED8C2DC6B604606B2B6064EF407DD77:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:26,237 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:26,238 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:26,238 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:26,238 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[68F324BD7F684E389FDF0862906B5F4F:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [EED8C2DC6B604606B2B6064EF407DD77:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:26,238 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:26,238 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 68F324BD7F684E389FDF0862906B5F4F being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:26,238 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 68F324BD7F684E389FDF0862906B5F4F
2016-05-18 02:07:26,238 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:26,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:26,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:26,239 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537246029_0001_1_00_000000_0
2016-05-18 02:07:26,241 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:26,241 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 68F324BD7F684E389FDF0862906B5F4F initialized RecordReader from event
2016-05-18 02:07:26,243 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:26,243 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 68F324BD7F684E389FDF0862906B5F4F
2016-05-18 02:07:26,243 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_-1"], id: EED8C2DC6B604606B2B6064EF407DD77
2016-05-18 02:07:26,244 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_-1"]
2016-05-18 02:07:26,244 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:26,244 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B4A75DA5A5A94AE8B208E398D2481672, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:26,246 INFO output.MROutput (MROutput.java:close(526)) - EED8C2DC6B604606B2B6064EF407DD77 closed
2016-05-18 02:07:26,247 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537246029_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:26,247 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537246029_0001_1_00_000000_0
2016-05-18 02:07:26,247 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B4A75DA5A5A94AE8B208E398D2481672, mem on close (mb), free: 199, total: 387, max: 672
2016-05-18 02:07:26,247 INFO output.MROutput (MROutput.java:close(526)) - EED8C2DC6B604606B2B6064EF407DD77 closed
2016-05-18 02:07:26,247 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:26 Completed running task attempt: attempt_1463537246029_0001_1_00_000000_0
2016-05-18 02:07:26,248 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537246029_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:26,248 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537246029_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:26,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537246029_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537246244, Process_Duration=3, Process_End_Time=1463537246247, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:26,250 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537246029_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:26,250 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:26,250 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246029_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B4A75DA5A5A94AE8B208E398D2481672, taskAttemptId=attempt_1463537246029_0001_1_00_000000_0, creationTime=1463537246215, allocationTime=1463537246217, startTime=1463537246219, finishTime=1463537246250, timeTaken=31, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:26,250 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537246029_0001_1_00_000000_0
2016-05-18 02:07:26,250 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:26,250 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:26,251 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537246029_0001_00_000001
2016-05-18 02:07:26,251 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246029_0001_1][Event:TASK_FINISHED]: vertexName=B4A75DA5A5A94AE8B208E398D2481672, taskId=task_1463537246029_0001_1_00_000000, startTime=1463537246219, finishTime=1463537246250, timeTaken=31, status=SUCCEEDED, successfulAttemptID=attempt_1463537246029_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537246244, Process_Duration=3, Process_End_Time=1463537246247, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:26,251 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:26,252 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672]
2016-05-18 02:07:26,252 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246029_0001_1][Event:VERTEX_FINISHED]: vertexName=B4A75DA5A5A94AE8B208E398D2481672, vertexId=vertex_1463537246029_0001_1_00, initRequestedTime=1463537246200, initedTime=1463537246210, startRequestedTime=1463537246202, startedTime=1463537246213, finishTime=1463537246252, timeTaken=39, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537246244, Process_Duration=3, Process_End_Time=1463537246247, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537246219, firstTasksToStart=[ task_1463537246029_0001_1_00_000000 ], lastTaskFinishTime=1463537246250, lastTasksToFinish=[ task_1463537246029_0001_1_00_000000 ], minTaskDuration=31, maxTaskDuration=31, avgTaskDuration=31.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537246029_0001_1_00_000000 ], longestDurationTasks=[ task_1463537246029_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:26,253 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:26,253 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246029_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537246029_0001_00_000001, stoppedTime=1463537246253, exitStatus=0
2016-05-18 02:07:26,253 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:26,254 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:26,254 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537246029_0001_1
2016-05-18 02:07:26,254 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246029_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537246029_0001_1
2016-05-18 02:07:26,254 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: EED8C2DC6B604606B2B6064EF407DD77 for vertex: vertex_1463537246029_0001_1_00 [B4A75DA5A5A94AE8B208E398D2481672], outputName: EED8C2DC6B604606B2B6064EF407DD77
2016-05-18 02:07:26,254 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537246029_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:26,255 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537246029_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:26,261 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537246029_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:26,261 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537246029_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:26,262 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537246029_0001_00_000001, Exiting
2016-05-18 02:07:26,262 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537246029_0001_00_000001
2016-05-18 02:07:26,262 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537246029_0001_00_000001
2016-05-18 02:07:26,262 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537246029_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:26,263 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:EED8C2DC6B604606B2B6064EF407DD77 of vertex/vertexGroup:B4A75DA5A5A94AE8B208E398D2481672 isVertexGroupOutput:false
2016-05-18 02:07:26,263 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:26,265 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537246029_0001_1, queueSize=0
2016-05-18 02:07:26,265 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246029_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537246029_0001_1, startTime=1463537246199, finishTime=1463537246263, timeTaken=64, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=120, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537246244, Process_Duration=3, Process_End_Time=1463537246247, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:26,265 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537246029_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:26,265 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537246029_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:26 Completed Dag: dag_1463537246029_0001_1
2016-05-18 02:07:26,266 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537246029_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:26,266 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537246029_0001_1_post
2016-05-18 02:07:26,266 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:26,266 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:26,266 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:26,266 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[719CBD48C9EB4E32BAFDA1D0C4341390/A182D403E8DD4A368067427C832C5AD6] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1, with id=dag_1463537246029_0001_1
2016-05-18 02:07:26,266 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[719CBD48C9EB4E32BAFDA1D0C4341390/A182D403E8DD4A368067427C832C5AD6] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1, with id=dag_1463537246029_0001_1
2016-05-18 02:07:26,267 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537246029_0001_00_000001 completed successfully
2016-05-18 02:07:26,271 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...'field'_RESULTS.0_ARGS_-1, applicationId=application_1463537246029_0001
2016-05-18 02:07:26,273 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:26,273 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:26,273 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:26,273 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:26,273 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:26,274 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:26,274 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:26,274 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:26,274 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:26,275 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:26,275 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:26,275 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:26,275 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:26,276 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:26,279 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/719CBD48C9EB4E32BAFDA1D0C4341390/A182D403E8DD4A368067427C832C5AD6/.tez/application_1463537246029_0001
2016-05-18 02:07:26,279 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_-1/_temporary
2016-05-18 02:07:26,280 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.271
2016-05-18 02:07:26,349 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_ARGS_-1
2016-05-18 02:07:26,375 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:26,375 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:26,406 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-18 02:07:26,408 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.006
2016-05-18 02:07:26,408 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 02:07:26,408 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 02:07:26,409 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_ARGS_'field2'
2016-05-18 02:07:26,432 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:26,432 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:26,464 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-18 02:07:26,469 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-18 02:07:26,470 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:26,470 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:26,471 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:26,487 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:26,488 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_RESULTS
2016-05-18 02:07:26,498 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:26,499 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:26,499 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_RESULTS"]
2016-05-18 02:07:26,499 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:26,499 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:26,500 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:26,500 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...d'_RESULTS.0_ARGS_RESULTS
2016-05-18 02:07:26,501 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:26,516 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:26,521 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:26,521 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:26,522 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:26,532 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/19E364D3671D4167A40F08A8948B34BA/C7E9E69C2955411B8811D9123D745C9B/.tez/application_1463537246521_0001 doesn't exist and is created
2016-05-18 02:07:26,550 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 02:07:26,550 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 02:07:26,557 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:26,557 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:26,557 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/19E364D3671D4167A40F08A8948B34BA/C7E9E69C2955411B8811D9123D745C9B/.tez/application_1463537246521_0001_wd
2016-05-18 02:07:26,585 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537246521_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:26,585 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:26,586 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:26,587 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:26,587 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:26,587 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:26,587 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:26,588 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:26,589 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:26,589 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:26,589 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:26,589 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:26,589 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:26,589 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:26,589 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:26,590 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:26,590 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:26,590 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:26,590 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:26,590 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:26,591 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:26,592 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:26,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:26,592 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/19E364D3671D4167A40F08A8948B34BA/C7E9E69C2955411B8811D9123D745C9B/.tez/application_1463537246521_0001_wd/localmode-log-dir
2016-05-18 02:07:26,592 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/19E364D3671D4167A40F08A8948B34BA/C7E9E69C2955411B8811D9123D745C9B/.tez/application_1463537246521_0001_wd/localmode-log-dir/history.txt.appattempt_1463537246521_0001_000000, maxErrors=10
2016-05-18 02:07:26,593 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/19E364D3671D4167A40F08A8948B34BA/C7E9E69C2955411B8811D9123D745C9B/.tez/application_1463537246521_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:26,593 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537246521_0001_000000, appSubmitTime=1463537246585, launchTime=1463537246585
2016-05-18 02:07:26,596 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:26,600 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:51771
2016-05-18 02:07:26,600 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537246521
2016-05-18 02:07:26,603 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537246521_0001_000000, startTime=1463537246603
2016-05-18 02:07:26,603 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:26,657 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:26,657 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:26,657 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...d'_RESULTS.0_ARGS_RESULTS, applicationId=application_1463537246521_0001, dagName=[19E364D3671D4167A40F08A8948B34BA/C7E9E69C2955411B8811D9123D745C9B] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS
2016-05-18 02:07:26,676 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [19E364D3671D4167A40F08A8948B34BA/C7E9E69C2955411B8811D9123D745C9B] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS
2016-05-18 02:07:26,677 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:26,678 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537246521_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/19E364D3671D4167A40F08A8948B34BA/C7E9E69C2955411B8811D9123D745C9B/.tez/application_1463537246521_0001_wd/localmode-log-dir/dag_1463537246521_0001_1.dot
2016-05-18 02:07:26,678 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/19E364D3671D4167A40F08A8948B34BA/C7E9E69C2955411B8811D9123D745C9B/.tez/application_1463537246521_0001_wd/localmode-log-dir/dag_1463537246521_0001_1-tez-dag.pb.txt
2016-05-18 02:07:26,682 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537246521_0001_1
2016-05-18 02:07:26,682 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:26,682 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [19E364D3671D4167A40F08A8948B34BA/C7E9E69C2955411B8811D9123D745C9B] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS
2016-05-18 02:07:26 Running Dag: dag_1463537246521_0001_1
2016-05-18 02:07:26,696 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246521_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537246521_0001_1, submitTime=1463537246677
2016-05-18 02:07:26,697 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/19E364D3671D4167A40F08A8948B34BA/C7E9E69C2955411B8811D9123D745C9B/.tez/application_1463537246521_0001/tez-conf.pb" } size: 40947 timestamp: 1463537246000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:26,698 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D]
2016-05-18 02:07:26,698 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D]
2016-05-18 02:07:26,699 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:26,699 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:26,699 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246521_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537246521_0001_1, initTime=1463537246699
2016-05-18 02:07:26,700 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537246521_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:26,700 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:26,700 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246521_0001_1][Event:DAG_STARTED]: dagID=dag_1463537246521_0001_1, startTime=1463537246700
2016-05-18 02:07:26,701 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537246521_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:26,701 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: BED65A3215224253B180D944E5107F0D : {9B991AE48B4D407C93ECF7509726B282={InputName=9B991AE48B4D407C93ECF7509726B282}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:26,701 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 9B991AE48B4D407C93ECF7509726B282, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:26,701 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D]
2016-05-18 02:07:26,701 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D]
2016-05-18 02:07:26,701 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D]
2016-05-18 02:07:26,702 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D]
2016-05-18 02:07:26,702 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:26,702 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...d'_RESULTS.0_ARGS_RESULTS, applicationId=application_1463537246521_0001, dagName=[19E364D3671D4167A40F08A8948B34BA/C7E9E69C2955411B8811D9123D745C9B] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS
2016-05-18 02:07:26,703 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 9B991AE48B4D407C93ECF7509726B282 on vertex vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D]
2016-05-18 02:07:26,705 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 9B991AE48B4D407C93ECF7509726B282 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:26,706 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:26,706 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:26,706 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:26,706 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:26,707 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 9B991AE48B4D407C93ECF7509726B282 on vertex vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D]
2016-05-18 02:07:26,708 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:26,708 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {9B991AE48B4D407C93ECF7509726B282=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:26,708 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D] parallelism set to 1
2016-05-18 02:07:26,708 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:26,709 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D], numAdditionalOutputs=1
2016-05-18 02:07:26,709 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=36247D3593D84479A2C2C9F356BEF881, vertex=vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:26,711 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for BED65A3215224253B180D944E5107F0D:36247D3593D84479A2C2C9F356BEF881 using oldmapred API
2016-05-18 02:07:26,711 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=36247D3593D84479A2C2C9F356BEF881, vertexName=BED65A3215224253B180D944E5107F0D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:26,712 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246521_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BED65A3215224253B180D944E5107F0D, vertexId=vertex_1463537246521_0001_1_00, initRequestedTime=1463537246701, initedTime=1463537246709, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:26,712 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D] with distanceFromRoot: 0
2016-05-18 02:07:26,712 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:26,713 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246521_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537246521_0001_1_00, startRequestedTime=1463537246702, startedTime=1463537246712
2016-05-18 02:07:26,713 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246521_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537246521_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:26,713 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:26,714 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in BED65A3215224253B180D944E5107F0D
2016-05-18 02:07:26,714 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D]
2016-05-18 02:07:26,714 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246521_0001_1][Event:TASK_STARTED]: vertexName=BED65A3215224253B180D944E5107F0D, taskId=task_1463537246521_0001_1_00_000000, scheduledTime=1463537246714, launchTime=1463537246714
2016-05-18 02:07:26,715 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D]
2016-05-18 02:07:26,716 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:26,717 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537246521_0001_00_000001, launchTime=1463537246717
2016-05-18 02:07:26,718 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537246521_0001_00_000001
2016-05-18 02:07:26,719 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537246521_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537246521_0001_1_00_000000_0
2016-05-18 02:07:26,719 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537246521_0001_1_00_000000_0] started. Is using containerId: [container_1463537246521_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:26,720 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246521_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BED65A3215224253B180D944E5107F0D, taskAttemptId=attempt_1463537246521_0001_1_00_000000_0, startTime=1463537246719, containerId=container_1463537246521_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:26 Starting to run new task attempt: attempt_1463537246521_0001_1_00_000000_0
2016-05-18 02:07:26,720 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537246521_0001_1_00_000000_0
2016-05-18 02:07:26,721 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:26,721 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:26,721 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [19E364D3671D4167A40F08A8948B34BA/C7E9E69C2955411B8811D9123D745C9B] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS, VertexName: BED65A3215224253B180D944E5107F0D, VertexParallelism: 1, TaskAttemptID:attempt_1463537246521_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9B991AE48B4D407C93ECF7509726B282, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=36247D3593D84479A2C2C9F356BEF881, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:26,721 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:26,722 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537246521_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:26,723 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537246521_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:26,723 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537246521_0001_1_00_000000_0
2016-05-18 02:07:26,724 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537246521_0001, with dag id: dag_1463537246521_0001_1
2016-05-18 02:07:26,727 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:26,727 INFO input.MRInput (MRInput.java:initialize(440)) - 9B991AE48B4D407C93ECF7509726B282 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:26,728 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 9B991AE48B4D407C93ECF7509726B282
2016-05-18 02:07:26,737 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BED65A3215224253B180D944E5107F0D, ordinal: 0
2016-05-18 02:07:26,737 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BED65A3215224253B180D944E5107F0D, mem on start (mb), free: 296, total: 388, max: 672
2016-05-18 02:07:26,737 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:26,737 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:26,738 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:26,741 INFO output.MROutput (MROutput.java:initialize(421)) - 36247D3593D84479A2C2C9F356BEF881: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:26,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:26,742 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[9B991AE48B4D407C93ECF7509726B282:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [36247D3593D84479A2C2C9F356BEF881:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:26,742 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:26,742 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:26,742 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:26,742 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[9B991AE48B4D407C93ECF7509726B282:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [36247D3593D84479A2C2C9F356BEF881:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:26,742 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:26,742 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 9B991AE48B4D407C93ECF7509726B282 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:26,743 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:26,743 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:26,743 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 9B991AE48B4D407C93ECF7509726B282
2016-05-18 02:07:26,743 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:26,744 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537246521_0001_1_00_000000_0
2016-05-18 02:07:26,748 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:26,748 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 9B991AE48B4D407C93ECF7509726B282
2016-05-18 02:07:26,749 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_RESULTS"], id: 36247D3593D84479A2C2C9F356BEF881
2016-05-18 02:07:26,749 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_RESULTS"]
2016-05-18 02:07:26,749 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:26,750 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:26,750 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 9B991AE48B4D407C93ECF7509726B282 initialized RecordReader from event
2016-05-18 02:07:26,750 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BED65A3215224253B180D944E5107F0D, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:26,754 INFO output.MROutput (MROutput.java:close(526)) - 36247D3593D84479A2C2C9F356BEF881 closed
2016-05-18 02:07:26,755 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537246521_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:26,755 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537246521_0001_1_00_000000_0
2016-05-18 02:07:26,756 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BED65A3215224253B180D944E5107F0D, mem on close (mb), free: 294, total: 388, max: 672
2016-05-18 02:07:26,756 INFO output.MROutput (MROutput.java:close(526)) - 36247D3593D84479A2C2C9F356BEF881 closed
2016-05-18 02:07:26,756 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:26 Completed running task attempt: attempt_1463537246521_0001_1_00_000000_0
2016-05-18 02:07:26,756 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537246521_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:26,757 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537246521_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:26,757 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537246521_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537246751, Process_Duration=4, Process_End_Time=1463537246755, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:26,758 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537246521_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:26,759 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:26,759 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246521_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BED65A3215224253B180D944E5107F0D, taskAttemptId=attempt_1463537246521_0001_1_00_000000_0, creationTime=1463537246715, allocationTime=1463537246717, startTime=1463537246719, finishTime=1463537246758, timeTaken=39, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:26,759 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537246521_0001_1_00_000000_0
2016-05-18 02:07:26,759 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:26,759 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:26,761 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246521_0001_1][Event:TASK_FINISHED]: vertexName=BED65A3215224253B180D944E5107F0D, taskId=task_1463537246521_0001_1_00_000000, startTime=1463537246719, finishTime=1463537246759, timeTaken=40, status=SUCCEEDED, successfulAttemptID=attempt_1463537246521_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537246751, Process_Duration=4, Process_End_Time=1463537246755, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:26,761 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:26,761 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D]
2016-05-18 02:07:26,762 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246521_0001_1][Event:VERTEX_FINISHED]: vertexName=BED65A3215224253B180D944E5107F0D, vertexId=vertex_1463537246521_0001_1_00, initRequestedTime=1463537246701, initedTime=1463537246709, startRequestedTime=1463537246702, startedTime=1463537246712, finishTime=1463537246761, timeTaken=49, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537246751, Process_Duration=4, Process_End_Time=1463537246755, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537246719, firstTasksToStart=[ task_1463537246521_0001_1_00_000000 ], lastTaskFinishTime=1463537246758, lastTasksToFinish=[ task_1463537246521_0001_1_00_000000 ], minTaskDuration=39, maxTaskDuration=39, avgTaskDuration=39.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537246521_0001_1_00_000000 ], longestDurationTasks=[ task_1463537246521_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:26,762 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:26,762 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246521_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537246521_0001_00_000001, stoppedTime=1463537246762, exitStatus=0
2016-05-18 02:07:26,763 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:26,763 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:26,763 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537246521_0001_1
2016-05-18 02:07:26,763 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246521_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537246521_0001_1
2016-05-18 02:07:26,763 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 36247D3593D84479A2C2C9F356BEF881 for vertex: vertex_1463537246521_0001_1_00 [BED65A3215224253B180D944E5107F0D], outputName: 36247D3593D84479A2C2C9F356BEF881
2016-05-18 02:07:26,763 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537246521_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:26,765 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537246521_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:26,765 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537246521_0001_00_000001
2016-05-18 02:07:26,765 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537246521_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:26,765 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537246521_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:26,765 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537246521_0001_00_000001, Exiting
2016-05-18 02:07:26,766 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537246521_0001_00_000001
2016-05-18 02:07:26,766 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537246521_0001_00_000001
2016-05-18 02:07:26,769 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537246521_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:26,770 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537246521_0001_00_000001 completed successfully
2016-05-18 02:07:26,775 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:36247D3593D84479A2C2C9F356BEF881 of vertex/vertexGroup:BED65A3215224253B180D944E5107F0D isVertexGroupOutput:false
2016-05-18 02:07:26,775 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:26,777 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537246521_0001_1, queueSize=0
2016-05-18 02:07:26,777 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537246521_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537246521_0001_1, startTime=1463537246700, finishTime=1463537246775, timeTaken=75, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=170, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537246751, Process_Duration=4, Process_End_Time=1463537246755, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:26,777 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537246521_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:26,778 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537246521_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:26 Completed Dag: dag_1463537246521_0001_1
2016-05-18 02:07:26,778 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537246521_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:26,778 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537246521_0001_1_post
2016-05-18 02:07:26,778 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:26,779 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:26,779 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:26,779 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[19E364D3671D4167A40F08A8948B34BA/C7E9E69C2955411B8811D9123D745C9B] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS, with id=dag_1463537246521_0001_1
2016-05-18 02:07:26,779 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[19E364D3671D4167A40F08A8948B34BA/C7E9E69C2955411B8811D9123D745C9B] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS, with id=dag_1463537246521_0001_1
2016-05-18 02:07:26,790 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...d'_RESULTS.0_ARGS_RESULTS, applicationId=application_1463537246521_0001
2016-05-18 02:07:26,792 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:26,792 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:26,792 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:26,792 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:26,792 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:26,793 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:26,793 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:26,794 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:26,794 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:26,794 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:26,794 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:26,798 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:26,799 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:26,799 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:26,804 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/19E364D3671D4167A40F08A8948B34BA/C7E9E69C2955411B8811D9123D745C9B/.tez/application_1463537246521_0001
2016-05-18 02:07:26,804 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_RESULTS/_temporary
2016-05-18 02:07:26,805 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.306
2016-05-18 02:07:26,877 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_ARGS_RESULTS
2016-05-18 02:07:26,902 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:26,902 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:26,932 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-18 02:07:26,934 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-05-18 02:07:26,934 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 02:07:26,934 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 02:07:26,935 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_ARGS_ALL
2016-05-18 02:07:26,960 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:26,961 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:26,996 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-18 02:07:27,000 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-18 02:07:27,001 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:27,001 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:27,001 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:27,018 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:27,018 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_-1
2016-05-18 02:07:27,028 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:27,028 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:27,029 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_-1"]
2016-05-18 02:07:27,029 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:27,029 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:27,029 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:27,030 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...'_RESULTS.ALL_'field2'_-1
2016-05-18 02:07:27,030 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:27,044 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:27,048 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:27,048 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:27,049 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:27,059 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 02:07:27,059 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 02:07:27,059 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/E10B942CEBF34CFCABB336450BCD692F/E30B1B17CC364CD5A751310C2317EFB2/.tez/application_1463537247048_0001 doesn't exist and is created
2016-05-18 02:07:27,080 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:27,080 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:27,081 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/E10B942CEBF34CFCABB336450BCD692F/E30B1B17CC364CD5A751310C2317EFB2/.tez/application_1463537247048_0001_wd
2016-05-18 02:07:27,095 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537247048_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:27,095 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:27,095 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:27,098 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:27,098 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:27,098 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:27,098 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:27,099 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:27,100 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:27,101 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:27,101 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:27,101 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:27,101 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:27,101 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:27,102 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:27,102 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:27,102 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:27,102 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:27,102 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:27,103 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:27,105 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:27,106 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:27,106 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:27,106 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/E10B942CEBF34CFCABB336450BCD692F/E30B1B17CC364CD5A751310C2317EFB2/.tez/application_1463537247048_0001_wd/localmode-log-dir
2016-05-18 02:07:27,106 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/E10B942CEBF34CFCABB336450BCD692F/E30B1B17CC364CD5A751310C2317EFB2/.tez/application_1463537247048_0001_wd/localmode-log-dir/history.txt.appattempt_1463537247048_0001_000000, maxErrors=10
2016-05-18 02:07:27,107 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/E10B942CEBF34CFCABB336450BCD692F/E30B1B17CC364CD5A751310C2317EFB2/.tez/application_1463537247048_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:27,107 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537247048_0001_000000, appSubmitTime=1463537247094, launchTime=1463537247095
2016-05-18 02:07:27,112 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:27,113 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:46397
2016-05-18 02:07:27,113 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537247048
2016-05-18 02:07:27,119 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537247048_0001_000000, startTime=1463537247119
2016-05-18 02:07:27,119 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:27,180 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:27,180 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:27,181 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...'_RESULTS.ALL_'field2'_-1, applicationId=application_1463537247048_0001, dagName=[E10B942CEBF34CFCABB336450BCD692F/E30B1B17CC364CD5A751310C2317EFB2] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1
2016-05-18 02:07:27,198 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [E10B942CEBF34CFCABB336450BCD692F/E30B1B17CC364CD5A751310C2317EFB2] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1
2016-05-18 02:07:27,198 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:27,199 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537247048_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/E10B942CEBF34CFCABB336450BCD692F/E30B1B17CC364CD5A751310C2317EFB2/.tez/application_1463537247048_0001_wd/localmode-log-dir/dag_1463537247048_0001_1.dot
2016-05-18 02:07:27,199 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/E10B942CEBF34CFCABB336450BCD692F/E30B1B17CC364CD5A751310C2317EFB2/.tez/application_1463537247048_0001_wd/localmode-log-dir/dag_1463537247048_0001_1-tez-dag.pb.txt
2016-05-18 02:07:27,203 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537247048_0001_1
2016-05-18 02:07:27,203 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:27,203 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [E10B942CEBF34CFCABB336450BCD692F/E30B1B17CC364CD5A751310C2317EFB2] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1
2016-05-18 02:07:27 Running Dag: dag_1463537247048_0001_1
2016-05-18 02:07:27,217 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247048_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537247048_0001_1, submitTime=1463537247198
2016-05-18 02:07:27,218 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/E10B942CEBF34CFCABB336450BCD692F/E30B1B17CC364CD5A751310C2317EFB2/.tez/application_1463537247048_0001/tez-conf.pb" } size: 40947 timestamp: 1463537247000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:27,218 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C]
2016-05-18 02:07:27,219 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C]
2016-05-18 02:07:27,219 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:27,219 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:27,219 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247048_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537247048_0001_1, initTime=1463537247219
2016-05-18 02:07:27,220 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537247048_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:27,220 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:27,220 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247048_0001_1][Event:DAG_STARTED]: dagID=dag_1463537247048_0001_1, startTime=1463537247220
2016-05-18 02:07:27,220 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537247048_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:27,220 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: FE6B719411D841FC8058C5D8D5B30A3C : {29A846AF262A40568938026E2F3C8EAD={InputName=29A846AF262A40568938026E2F3C8EAD}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:27,221 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 29A846AF262A40568938026E2F3C8EAD, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:27,221 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C]
2016-05-18 02:07:27,221 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C]
2016-05-18 02:07:27,221 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C]
2016-05-18 02:07:27,221 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...'_RESULTS.ALL_'field2'_-1, applicationId=application_1463537247048_0001, dagName=[E10B942CEBF34CFCABB336450BCD692F/E30B1B17CC364CD5A751310C2317EFB2] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1
2016-05-18 02:07:27,221 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C]
2016-05-18 02:07:27,222 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:27,222 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 29A846AF262A40568938026E2F3C8EAD on vertex vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C]
2016-05-18 02:07:27,224 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 29A846AF262A40568938026E2F3C8EAD asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:27,226 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:27,226 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:27,226 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:27,226 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:27,227 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 29A846AF262A40568938026E2F3C8EAD on vertex vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C]
2016-05-18 02:07:27,228 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:27,228 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {29A846AF262A40568938026E2F3C8EAD=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:27,228 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C] parallelism set to 1
2016-05-18 02:07:27,228 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:27,229 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C], numAdditionalOutputs=1
2016-05-18 02:07:27,229 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=557E6ECE6F3847079DFCEB312C886EC0, vertex=vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:27,231 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for FE6B719411D841FC8058C5D8D5B30A3C:557E6ECE6F3847079DFCEB312C886EC0 using oldmapred API
2016-05-18 02:07:27,231 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=557E6ECE6F3847079DFCEB312C886EC0, vertexName=FE6B719411D841FC8058C5D8D5B30A3C, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:27,231 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247048_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FE6B719411D841FC8058C5D8D5B30A3C, vertexId=vertex_1463537247048_0001_1_00, initRequestedTime=1463537247220, initedTime=1463537247229, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:27,232 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C] with distanceFromRoot: 0
2016-05-18 02:07:27,232 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:27,232 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247048_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537247048_0001_1_00, startRequestedTime=1463537247222, startedTime=1463537247232
2016-05-18 02:07:27,232 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247048_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537247048_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:27,232 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:27,233 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in FE6B719411D841FC8058C5D8D5B30A3C
2016-05-18 02:07:27,233 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C]
2016-05-18 02:07:27,233 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247048_0001_1][Event:TASK_STARTED]: vertexName=FE6B719411D841FC8058C5D8D5B30A3C, taskId=task_1463537247048_0001_1_00_000000, scheduledTime=1463537247233, launchTime=1463537247233
2016-05-18 02:07:27,233 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C]
2016-05-18 02:07:27,234 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:27,236 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537247048_0001_00_000001, launchTime=1463537247236
2016-05-18 02:07:27,236 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537247048_0001_00_000001
2016-05-18 02:07:27,237 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537247048_0001_1_00_000000_0] started. Is using containerId: [container_1463537247048_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:27,237 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247048_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FE6B719411D841FC8058C5D8D5B30A3C, taskAttemptId=attempt_1463537247048_0001_1_00_000000_0, startTime=1463537247237, containerId=container_1463537247048_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:27,238 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537247048_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537247048_0001_1_00_000000_0
2016-05-18 02:07:27 Starting to run new task attempt: attempt_1463537247048_0001_1_00_000000_0
2016-05-18 02:07:27,238 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537247048_0001_1_00_000000_0
2016-05-18 02:07:27,239 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:27,239 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:27,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E10B942CEBF34CFCABB336450BCD692F/E30B1B17CC364CD5A751310C2317EFB2] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1, VertexName: FE6B719411D841FC8058C5D8D5B30A3C, VertexParallelism: 1, TaskAttemptID:attempt_1463537247048_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=29A846AF262A40568938026E2F3C8EAD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=557E6ECE6F3847079DFCEB312C886EC0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:27,239 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:27,240 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537247048_0001_1_00_000000_0
2016-05-18 02:07:27,240 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537247048_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:27,241 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537247048_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:27,242 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537247048_0001, with dag id: dag_1463537247048_0001_1
2016-05-18 02:07:27,247 INFO input.MRInput (MRInput.java:initialize(440)) - 29A846AF262A40568938026E2F3C8EAD using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:27,247 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:27,249 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 29A846AF262A40568938026E2F3C8EAD
2016-05-18 02:07:27,254 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FE6B719411D841FC8058C5D8D5B30A3C, ordinal: 0
2016-05-18 02:07:27,254 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FE6B719411D841FC8058C5D8D5B30A3C, mem on start (mb), free: 180, total: 388, max: 672
2016-05-18 02:07:27,254 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:27,255 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:27,255 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:27,257 INFO output.MROutput (MROutput.java:initialize(421)) - 557E6ECE6F3847079DFCEB312C886EC0: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:27,257 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:27,257 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[29A846AF262A40568938026E2F3C8EAD:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [557E6ECE6F3847079DFCEB312C886EC0:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:27,258 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:27,258 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:27,259 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:27,259 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[29A846AF262A40568938026E2F3C8EAD:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [557E6ECE6F3847079DFCEB312C886EC0:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:27,259 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:27,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 29A846AF262A40568938026E2F3C8EAD
2016-05-18 02:07:27,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 29A846AF262A40568938026E2F3C8EAD being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:27,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:27,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:27,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:27,261 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537247048_0001_1_00_000000_0
2016-05-18 02:07:27,262 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:27,262 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 29A846AF262A40568938026E2F3C8EAD initialized RecordReader from event
2016-05-18 02:07:27,264 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:27,264 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 29A846AF262A40568938026E2F3C8EAD
2016-05-18 02:07:27,264 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_-1"], id: 557E6ECE6F3847079DFCEB312C886EC0
2016-05-18 02:07:27,265 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_-1"]
2016-05-18 02:07:27,265 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:27,265 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FE6B719411D841FC8058C5D8D5B30A3C, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:27,267 INFO output.MROutput (MROutput.java:close(526)) - 557E6ECE6F3847079DFCEB312C886EC0 closed
2016-05-18 02:07:27,268 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537247048_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:27,268 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537247048_0001_1_00_000000_0
2016-05-18 02:07:27,268 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FE6B719411D841FC8058C5D8D5B30A3C, mem on close (mb), free: 178, total: 388, max: 672
2016-05-18 02:07:27,269 INFO output.MROutput (MROutput.java:close(526)) - 557E6ECE6F3847079DFCEB312C886EC0 closed
2016-05-18 02:07:27,269 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:27 Completed running task attempt: attempt_1463537247048_0001_1_00_000000_0
2016-05-18 02:07:27,269 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537247048_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:27,269 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537247048_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:27,270 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537247048_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537247265, Process_Duration=3, Process_End_Time=1463537247268, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:27,270 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537247048_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:27,270 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:27,271 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:27,271 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:27,271 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537247048_0001_00_000001
2016-05-18 02:07:27,271 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247048_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FE6B719411D841FC8058C5D8D5B30A3C, taskAttemptId=attempt_1463537247048_0001_1_00_000000_0, creationTime=1463537247233, allocationTime=1463537247235, startTime=1463537247237, finishTime=1463537247271, timeTaken=34, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:27,271 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537247048_0001_1_00_000000_0
2016-05-18 02:07:27,272 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247048_0001_1][Event:TASK_FINISHED]: vertexName=FE6B719411D841FC8058C5D8D5B30A3C, taskId=task_1463537247048_0001_1_00_000000, startTime=1463537247237, finishTime=1463537247272, timeTaken=35, status=SUCCEEDED, successfulAttemptID=attempt_1463537247048_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537247265, Process_Duration=3, Process_End_Time=1463537247268, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:27,272 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:27,273 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C]
2016-05-18 02:07:27,273 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247048_0001_1][Event:VERTEX_FINISHED]: vertexName=FE6B719411D841FC8058C5D8D5B30A3C, vertexId=vertex_1463537247048_0001_1_00, initRequestedTime=1463537247220, initedTime=1463537247229, startRequestedTime=1463537247222, startedTime=1463537247232, finishTime=1463537247273, timeTaken=41, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537247265, Process_Duration=3, Process_End_Time=1463537247268, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537247237, firstTasksToStart=[ task_1463537247048_0001_1_00_000000 ], lastTaskFinishTime=1463537247271, lastTasksToFinish=[ task_1463537247048_0001_1_00_000000 ], minTaskDuration=34, maxTaskDuration=34, avgTaskDuration=34.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537247048_0001_1_00_000000 ], longestDurationTasks=[ task_1463537247048_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:27,274 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:27,274 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247048_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537247048_0001_00_000001, stoppedTime=1463537247274, exitStatus=0
2016-05-18 02:07:27,274 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:27,274 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:27,274 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537247048_0001_1
2016-05-18 02:07:27,275 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247048_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537247048_0001_1
2016-05-18 02:07:27,275 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 557E6ECE6F3847079DFCEB312C886EC0 for vertex: vertex_1463537247048_0001_1_00 [FE6B719411D841FC8058C5D8D5B30A3C], outputName: 557E6ECE6F3847079DFCEB312C886EC0
2016-05-18 02:07:27,275 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537247048_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:27,275 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537247048_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:27,281 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537247048_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:27,281 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537247048_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:27,282 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537247048_0001_00_000001, Exiting
2016-05-18 02:07:27,282 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537247048_0001_00_000001
2016-05-18 02:07:27,282 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537247048_0001_00_000001
2016-05-18 02:07:27,282 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537247048_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:27,283 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537247048_0001_00_000001 completed successfully
2016-05-18 02:07:27,284 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:557E6ECE6F3847079DFCEB312C886EC0 of vertex/vertexGroup:FE6B719411D841FC8058C5D8D5B30A3C isVertexGroupOutput:false
2016-05-18 02:07:27,284 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:27,285 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537247048_0001_1, queueSize=0
2016-05-18 02:07:27,286 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247048_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537247048_0001_1, startTime=1463537247220, finishTime=1463537247284, timeTaken=64, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=110, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537247265, Process_Duration=3, Process_End_Time=1463537247268, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:27,286 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537247048_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:27,286 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537247048_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:27 Completed Dag: dag_1463537247048_0001_1
2016-05-18 02:07:27,287 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537247048_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:27,287 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537247048_0001_1_post
2016-05-18 02:07:27,287 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:27,287 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:27,288 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:27,288 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[E10B942CEBF34CFCABB336450BCD692F/E30B1B17CC364CD5A751310C2317EFB2] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1, with id=dag_1463537247048_0001_1
2016-05-18 02:07:27,288 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[E10B942CEBF34CFCABB336450BCD692F/E30B1B17CC364CD5A751310C2317EFB2] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1, with id=dag_1463537247048_0001_1
2016-05-18 02:07:27,293 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...'_RESULTS.ALL_'field2'_-1, applicationId=application_1463537247048_0001
2016-05-18 02:07:27,294 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:27,294 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:27,294 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:27,294 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:27,294 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:27,295 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:27,295 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:27,295 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:27,295 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:27,296 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:27,296 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:27,296 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:27,297 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:27,297 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:27,300 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/E10B942CEBF34CFCABB336450BCD692F/E30B1B17CC364CD5A751310C2317EFB2/.tez/application_1463537247048_0001
2016-05-18 02:07:27,301 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_-1/_temporary
2016-05-18 02:07:27,301 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.273
2016-05-18 02:07:27,372 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_'field2'_-1
2016-05-18 02:07:27,395 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:27,395 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:27,428 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-18 02:07:27,436 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-18 02:07:27,436 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:27,437 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:27,437 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:27,453 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:27,454 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field2'
2016-05-18 02:07:27,464 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:27,464 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:27,464 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field2'"]
2016-05-18 02:07:27,464 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:27,464 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:27,465 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:27,465 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...LTS.ALL_'field2'_'field2'
2016-05-18 02:07:27,466 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:27,480 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:27,484 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:27,485 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:27,485 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:27,493 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/80F99A059BBC4554B66679264DBAC7C6/E08F84D5DEA2416FB2BF0BDFDDB14E59/.tez/application_1463537247484_0001 doesn't exist and is created
2016-05-18 02:07:27,513 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:27,513 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:27,513 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/80F99A059BBC4554B66679264DBAC7C6/E08F84D5DEA2416FB2BF0BDFDDB14E59/.tez/application_1463537247484_0001_wd
2016-05-18 02:07:27,529 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537247484_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:27,529 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:27,529 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:27,530 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:27,530 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:27,530 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:27,530 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:27,531 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:27,543 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:27,544 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:27,544 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:27,544 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:27,544 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:27,544 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:27,544 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:27,544 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:27,544 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:27,544 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:27,545 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:27,545 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:27,547 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:27,547 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:27,547 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:27,548 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/80F99A059BBC4554B66679264DBAC7C6/E08F84D5DEA2416FB2BF0BDFDDB14E59/.tez/application_1463537247484_0001_wd/localmode-log-dir
2016-05-18 02:07:27,548 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/80F99A059BBC4554B66679264DBAC7C6/E08F84D5DEA2416FB2BF0BDFDDB14E59/.tez/application_1463537247484_0001_wd/localmode-log-dir/history.txt.appattempt_1463537247484_0001_000000, maxErrors=10
2016-05-18 02:07:27,548 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/80F99A059BBC4554B66679264DBAC7C6/E08F84D5DEA2416FB2BF0BDFDDB14E59/.tez/application_1463537247484_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:27,549 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537247484_0001_000000, appSubmitTime=1463537247528, launchTime=1463537247528
2016-05-18 02:07:27,552 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:27,557 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:46294
2016-05-18 02:07:27,558 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537247484
2016-05-18 02:07:27,560 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537247484_0001_000000, startTime=1463537247560
2016-05-18 02:07:27,560 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:27,575 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 02:07:27,575 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 02:07:27,614 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:27,614 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:27,614 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...LTS.ALL_'field2'_'field2', applicationId=application_1463537247484_0001, dagName=[80F99A059BBC4554B66679264DBAC7C6/E08F84D5DEA2416FB2BF0BDFDDB14E59] every.every/(1/1) ...LTS.ALL_'field2'_'field2'
2016-05-18 02:07:27,632 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [80F99A059BBC4554B66679264DBAC7C6/E08F84D5DEA2416FB2BF0BDFDDB14E59] every.every/(1/1) ...LTS.ALL_'field2'_'field2'
2016-05-18 02:07:27,632 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:27,633 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537247484_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/80F99A059BBC4554B66679264DBAC7C6/E08F84D5DEA2416FB2BF0BDFDDB14E59/.tez/application_1463537247484_0001_wd/localmode-log-dir/dag_1463537247484_0001_1.dot
2016-05-18 02:07:27,633 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/80F99A059BBC4554B66679264DBAC7C6/E08F84D5DEA2416FB2BF0BDFDDB14E59/.tez/application_1463537247484_0001_wd/localmode-log-dir/dag_1463537247484_0001_1-tez-dag.pb.txt
2016-05-18 02:07:27,637 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537247484_0001_1
2016-05-18 02:07:27,637 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:27,638 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [80F99A059BBC4554B66679264DBAC7C6/E08F84D5DEA2416FB2BF0BDFDDB14E59] every.every/(1/1) ...LTS.ALL_'field2'_'field2'
2016-05-18 02:07:27 Running Dag: dag_1463537247484_0001_1
2016-05-18 02:07:27,653 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247484_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537247484_0001_1, submitTime=1463537247632
2016-05-18 02:07:27,654 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/80F99A059BBC4554B66679264DBAC7C6/E08F84D5DEA2416FB2BF0BDFDDB14E59/.tez/application_1463537247484_0001/tez-conf.pb" } size: 40947 timestamp: 1463537247000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:27,654 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4]
2016-05-18 02:07:27,654 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4]
2016-05-18 02:07:27,655 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:27,655 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:27,656 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247484_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537247484_0001_1, initTime=1463537247655
2016-05-18 02:07:27,656 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537247484_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:27,656 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:27,657 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247484_0001_1][Event:DAG_STARTED]: dagID=dag_1463537247484_0001_1, startTime=1463537247656
2016-05-18 02:07:27,657 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537247484_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:27,657 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: BA3CA07357FF4209B0F940E1B8997DE4 : {59E3F672BA7749E28AC1769FBBFD4205={InputName=59E3F672BA7749E28AC1769FBBFD4205}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:27,657 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 59E3F672BA7749E28AC1769FBBFD4205, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:27,657 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4]
2016-05-18 02:07:27,657 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4]
2016-05-18 02:07:27,657 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4]
2016-05-18 02:07:27,657 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4]
2016-05-18 02:07:27,658 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:27,658 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...LTS.ALL_'field2'_'field2', applicationId=application_1463537247484_0001, dagName=[80F99A059BBC4554B66679264DBAC7C6/E08F84D5DEA2416FB2BF0BDFDDB14E59] every.every/(1/1) ...LTS.ALL_'field2'_'field2'
2016-05-18 02:07:27,659 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 59E3F672BA7749E28AC1769FBBFD4205 on vertex vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4]
2016-05-18 02:07:27,660 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 59E3F672BA7749E28AC1769FBBFD4205 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:27,662 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:27,662 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:27,662 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:27,662 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:27,663 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 59E3F672BA7749E28AC1769FBBFD4205 on vertex vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4]
2016-05-18 02:07:27,667 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:27,667 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {59E3F672BA7749E28AC1769FBBFD4205=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:27,667 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4] parallelism set to 1
2016-05-18 02:07:27,668 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:27,668 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4], numAdditionalOutputs=1
2016-05-18 02:07:27,668 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=2995884C1D994A769933845040974C19, vertex=vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:27,670 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for BA3CA07357FF4209B0F940E1B8997DE4:2995884C1D994A769933845040974C19 using oldmapred API
2016-05-18 02:07:27,671 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=2995884C1D994A769933845040974C19, vertexName=BA3CA07357FF4209B0F940E1B8997DE4, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:27,672 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247484_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BA3CA07357FF4209B0F940E1B8997DE4, vertexId=vertex_1463537247484_0001_1_00, initRequestedTime=1463537247657, initedTime=1463537247668, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:27,672 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4] with distanceFromRoot: 0
2016-05-18 02:07:27,672 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:27,672 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247484_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537247484_0001_1_00, startRequestedTime=1463537247658, startedTime=1463537247672
2016-05-18 02:07:27,673 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247484_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537247484_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:27,673 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:27,673 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in BA3CA07357FF4209B0F940E1B8997DE4
2016-05-18 02:07:27,673 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4]
2016-05-18 02:07:27,674 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247484_0001_1][Event:TASK_STARTED]: vertexName=BA3CA07357FF4209B0F940E1B8997DE4, taskId=task_1463537247484_0001_1_00_000000, scheduledTime=1463537247673, launchTime=1463537247673
2016-05-18 02:07:27,674 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4]
2016-05-18 02:07:27,675 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:27,676 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537247484_0001_00_000001, launchTime=1463537247676
2016-05-18 02:07:27,677 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537247484_0001_00_000001
2016-05-18 02:07:27,678 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537247484_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537247484_0001_1_00_000000_0
2016-05-18 02:07:27,678 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537247484_0001_1_00_000000_0] started. Is using containerId: [container_1463537247484_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:27,678 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247484_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BA3CA07357FF4209B0F940E1B8997DE4, taskAttemptId=attempt_1463537247484_0001_1_00_000000_0, startTime=1463537247678, containerId=container_1463537247484_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:27 Starting to run new task attempt: attempt_1463537247484_0001_1_00_000000_0
2016-05-18 02:07:27,679 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537247484_0001_1_00_000000_0
2016-05-18 02:07:27,679 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:27,679 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:27,680 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [80F99A059BBC4554B66679264DBAC7C6/E08F84D5DEA2416FB2BF0BDFDDB14E59] every.every/(1/1) ...LTS.ALL_'field2'_'field2', VertexName: BA3CA07357FF4209B0F940E1B8997DE4, VertexParallelism: 1, TaskAttemptID:attempt_1463537247484_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=59E3F672BA7749E28AC1769FBBFD4205, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=2995884C1D994A769933845040974C19, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:27,680 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:27,681 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537247484_0001_1_00_000000_0
2016-05-18 02:07:27,681 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537247484_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:27,682 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537247484_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:27,684 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537247484_0001, with dag id: dag_1463537247484_0001_1
2016-05-18 02:07:27,684 INFO input.MRInput (MRInput.java:initialize(440)) - 59E3F672BA7749E28AC1769FBBFD4205 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:27,684 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 59E3F672BA7749E28AC1769FBBFD4205
2016-05-18 02:07:27,685 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:27,692 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BA3CA07357FF4209B0F940E1B8997DE4, ordinal: 0
2016-05-18 02:07:27,692 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BA3CA07357FF4209B0F940E1B8997DE4, mem on start (mb), free: 297, total: 388, max: 672
2016-05-18 02:07:27,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:27,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:27,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:27,696 INFO output.MROutput (MROutput.java:initialize(421)) - 2995884C1D994A769933845040974C19: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:27,697 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:27,697 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[59E3F672BA7749E28AC1769FBBFD4205:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [2995884C1D994A769933845040974C19:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:27,697 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:27,697 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:27,697 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:27,697 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[59E3F672BA7749E28AC1769FBBFD4205:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [2995884C1D994A769933845040974C19:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:27,698 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:27,698 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 59E3F672BA7749E28AC1769FBBFD4205 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:27,698 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 59E3F672BA7749E28AC1769FBBFD4205
2016-05-18 02:07:27,698 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:27,698 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:27,698 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:27,698 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537247484_0001_1_00_000000_0
2016-05-18 02:07:27,700 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:27,700 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 59E3F672BA7749E28AC1769FBBFD4205 initialized RecordReader from event
2016-05-18 02:07:27,702 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:27,702 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 59E3F672BA7749E28AC1769FBBFD4205
2016-05-18 02:07:27,702 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field2'"], id: 2995884C1D994A769933845040974C19
2016-05-18 02:07:27,703 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field2'"]
2016-05-18 02:07:27,703 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:27,703 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BA3CA07357FF4209B0F940E1B8997DE4, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:27,706 INFO output.MROutput (MROutput.java:close(526)) - 2995884C1D994A769933845040974C19 closed
2016-05-18 02:07:27,706 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537247484_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:27,707 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537247484_0001_1_00_000000_0
2016-05-18 02:07:27,707 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BA3CA07357FF4209B0F940E1B8997DE4, mem on close (mb), free: 295, total: 388, max: 672
2016-05-18 02:07:27,707 INFO output.MROutput (MROutput.java:close(526)) - 2995884C1D994A769933845040974C19 closed
2016-05-18 02:07:27,707 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:27 Completed running task attempt: attempt_1463537247484_0001_1_00_000000_0
2016-05-18 02:07:27,708 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537247484_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:27,708 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537247484_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:27,709 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537247484_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537247703, Process_Duration=4, Process_End_Time=1463537247707, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:27,710 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537247484_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:27,710 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:27,710 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247484_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BA3CA07357FF4209B0F940E1B8997DE4, taskAttemptId=attempt_1463537247484_0001_1_00_000000_0, creationTime=1463537247674, allocationTime=1463537247675, startTime=1463537247678, finishTime=1463537247710, timeTaken=32, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:27,710 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:27,710 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:27,710 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537247484_0001_1_00_000000_0
2016-05-18 02:07:27,710 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537247484_0001_00_000001
2016-05-18 02:07:27,711 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247484_0001_1][Event:TASK_FINISHED]: vertexName=BA3CA07357FF4209B0F940E1B8997DE4, taskId=task_1463537247484_0001_1_00_000000, startTime=1463537247678, finishTime=1463537247710, timeTaken=32, status=SUCCEEDED, successfulAttemptID=attempt_1463537247484_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537247703, Process_Duration=4, Process_End_Time=1463537247707, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:27,711 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:27,711 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4]
2016-05-18 02:07:27,712 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247484_0001_1][Event:VERTEX_FINISHED]: vertexName=BA3CA07357FF4209B0F940E1B8997DE4, vertexId=vertex_1463537247484_0001_1_00, initRequestedTime=1463537247657, initedTime=1463537247668, startRequestedTime=1463537247658, startedTime=1463537247672, finishTime=1463537247711, timeTaken=39, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537247703, Process_Duration=4, Process_End_Time=1463537247707, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537247678, firstTasksToStart=[ task_1463537247484_0001_1_00_000000 ], lastTaskFinishTime=1463537247710, lastTasksToFinish=[ task_1463537247484_0001_1_00_000000 ], minTaskDuration=32, maxTaskDuration=32, avgTaskDuration=32.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537247484_0001_1_00_000000 ], longestDurationTasks=[ task_1463537247484_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:27,712 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:27,712 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:27,713 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:27,713 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537247484_0001_1
2016-05-18 02:07:27,713 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247484_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537247484_0001_1
2016-05-18 02:07:27,713 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 2995884C1D994A769933845040974C19 for vertex: vertex_1463537247484_0001_1_00 [BA3CA07357FF4209B0F940E1B8997DE4], outputName: 2995884C1D994A769933845040974C19
2016-05-18 02:07:27,713 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537247484_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:27,717 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247484_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537247484_0001_00_000001, stoppedTime=1463537247717, exitStatus=0
2016-05-18 02:07:27,717 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537247484_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:27,721 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537247484_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:27,721 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537247484_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:27,721 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537247484_0001_00_000001, Exiting
2016-05-18 02:07:27,722 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537247484_0001_00_000001
2016-05-18 02:07:27,722 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537247484_0001_00_000001
2016-05-18 02:07:27,722 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537247484_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:27,722 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537247484_0001_00_000001 completed successfully
2016-05-18 02:07:27,725 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:2995884C1D994A769933845040974C19 of vertex/vertexGroup:BA3CA07357FF4209B0F940E1B8997DE4 isVertexGroupOutput:false
2016-05-18 02:07:27,725 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:27,726 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537247484_0001_1, queueSize=0
2016-05-18 02:07:27,727 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247484_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537247484_0001_1, startTime=1463537247656, finishTime=1463537247725, timeTaken=69, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=120, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537247703, Process_Duration=4, Process_End_Time=1463537247707, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:27,727 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537247484_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:27,727 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537247484_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:27 Completed Dag: dag_1463537247484_0001_1
2016-05-18 02:07:27,727 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537247484_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:27,727 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537247484_0001_1_post
2016-05-18 02:07:27,727 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:27,727 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:27,728 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:27,728 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[80F99A059BBC4554B66679264DBAC7C6/E08F84D5DEA2416FB2BF0BDFDDB14E59] every.every/(1/1) ...LTS.ALL_'field2'_'field2', with id=dag_1463537247484_0001_1
2016-05-18 02:07:27,728 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[80F99A059BBC4554B66679264DBAC7C6/E08F84D5DEA2416FB2BF0BDFDDB14E59] every.every/(1/1) ...LTS.ALL_'field2'_'field2', with id=dag_1463537247484_0001_1
2016-05-18 02:07:27,734 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...LTS.ALL_'field2'_'field2', applicationId=application_1463537247484_0001
2016-05-18 02:07:27,736 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:27,736 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:27,736 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:27,736 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:27,736 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:27,737 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:27,737 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:27,737 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:27,737 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:27,737 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:27,738 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:27,738 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:27,738 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:27,739 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:27,742 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/80F99A059BBC4554B66679264DBAC7C6/E08F84D5DEA2416FB2BF0BDFDDB14E59/.tez/application_1463537247484_0001
2016-05-18 02:07:27,743 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field2'/_temporary
2016-05-18 02:07:27,743 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.278
2016-05-18 02:07:27,812 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_'field2'_'field2'
2016-05-18 02:07:27,836 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:27,836 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:27,873 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-18 02:07:27,879 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-18 02:07:27,880 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:27,880 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:27,880 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:27,895 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:27,896 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field'
2016-05-18 02:07:27,907 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:27,908 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:27,908 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field'"]
2016-05-18 02:07:27,908 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:27,908 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:27,909 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:27,909 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ULTS.ALL_'field2'_'field'
2016-05-18 02:07:27,910 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:27,923 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:27,927 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:27,928 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:27,928 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:27,936 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/D87B42EF1B274C6D9FCB21BCCF6B7380/F80A5FAB7B0649F19B8C5AD73BB7DADA/.tez/application_1463537247927_0001 doesn't exist and is created
2016-05-18 02:07:27,957 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:27,957 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:27,958 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/D87B42EF1B274C6D9FCB21BCCF6B7380/F80A5FAB7B0649F19B8C5AD73BB7DADA/.tez/application_1463537247927_0001_wd
2016-05-18 02:07:27,975 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537247927_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:27,975 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:27,975 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:27,977 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:27,977 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:27,977 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:27,977 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:27,978 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:27,979 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:27,980 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:27,980 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:27,980 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:27,980 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:27,980 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:27,980 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:27,980 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:27,980 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:27,980 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:27,980 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:27,981 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:27,982 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:27,983 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:27,984 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:27,984 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/D87B42EF1B274C6D9FCB21BCCF6B7380/F80A5FAB7B0649F19B8C5AD73BB7DADA/.tez/application_1463537247927_0001_wd/localmode-log-dir
2016-05-18 02:07:27,984 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/D87B42EF1B274C6D9FCB21BCCF6B7380/F80A5FAB7B0649F19B8C5AD73BB7DADA/.tez/application_1463537247927_0001_wd/localmode-log-dir/history.txt.appattempt_1463537247927_0001_000000, maxErrors=10
2016-05-18 02:07:27,984 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/D87B42EF1B274C6D9FCB21BCCF6B7380/F80A5FAB7B0649F19B8C5AD73BB7DADA/.tez/application_1463537247927_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:27,985 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537247927_0001_000000, appSubmitTime=1463537247975, launchTime=1463537247975
2016-05-18 02:07:27,987 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:27,992 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:53497
2016-05-18 02:07:27,992 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537247927
2016-05-18 02:07:27,994 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537247927_0001_000000, startTime=1463537247994
2016-05-18 02:07:27,994 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:28,050 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 02:07:28,050 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 02:07:28,057 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:28,058 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:28,058 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ULTS.ALL_'field2'_'field', applicationId=application_1463537247927_0001, dagName=[D87B42EF1B274C6D9FCB21BCCF6B7380/F80A5FAB7B0649F19B8C5AD73BB7DADA] every.every/(1/1) ...ULTS.ALL_'field2'_'field'
2016-05-18 02:07:28,075 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [D87B42EF1B274C6D9FCB21BCCF6B7380/F80A5FAB7B0649F19B8C5AD73BB7DADA] every.every/(1/1) ...ULTS.ALL_'field2'_'field'
2016-05-18 02:07:28,075 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:28,076 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537247927_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/D87B42EF1B274C6D9FCB21BCCF6B7380/F80A5FAB7B0649F19B8C5AD73BB7DADA/.tez/application_1463537247927_0001_wd/localmode-log-dir/dag_1463537247927_0001_1.dot
2016-05-18 02:07:28,077 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/D87B42EF1B274C6D9FCB21BCCF6B7380/F80A5FAB7B0649F19B8C5AD73BB7DADA/.tez/application_1463537247927_0001_wd/localmode-log-dir/dag_1463537247927_0001_1-tez-dag.pb.txt
2016-05-18 02:07:28,080 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537247927_0001_1
2016-05-18 02:07:28,080 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:28,080 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [D87B42EF1B274C6D9FCB21BCCF6B7380/F80A5FAB7B0649F19B8C5AD73BB7DADA] every.every/(1/1) ...ULTS.ALL_'field2'_'field'
2016-05-18 02:07:28 Running Dag: dag_1463537247927_0001_1
2016-05-18 02:07:28,095 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247927_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537247927_0001_1, submitTime=1463537248075
2016-05-18 02:07:28,096 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/D87B42EF1B274C6D9FCB21BCCF6B7380/F80A5FAB7B0649F19B8C5AD73BB7DADA/.tez/application_1463537247927_0001/tez-conf.pb" } size: 40947 timestamp: 1463537247000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:28,096 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573]
2016-05-18 02:07:28,096 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573]
2016-05-18 02:07:28,096 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:28,097 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:28,097 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247927_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537247927_0001_1, initTime=1463537248097
2016-05-18 02:07:28,097 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537247927_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:28,098 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:28,098 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247927_0001_1][Event:DAG_STARTED]: dagID=dag_1463537247927_0001_1, startTime=1463537248098
2016-05-18 02:07:28,098 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537247927_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:28,098 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: F758771F2AE14F7A970DF14D4E08C573 : {5D9D05BC43A24444A35D36822985BE74={InputName=5D9D05BC43A24444A35D36822985BE74}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:28,098 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ULTS.ALL_'field2'_'field', applicationId=application_1463537247927_0001, dagName=[D87B42EF1B274C6D9FCB21BCCF6B7380/F80A5FAB7B0649F19B8C5AD73BB7DADA] every.every/(1/1) ...ULTS.ALL_'field2'_'field'
2016-05-18 02:07:28,098 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 5D9D05BC43A24444A35D36822985BE74, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:28,099 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573]
2016-05-18 02:07:28,099 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573]
2016-05-18 02:07:28,099 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573]
2016-05-18 02:07:28,099 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573]
2016-05-18 02:07:28,100 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:28,100 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 5D9D05BC43A24444A35D36822985BE74 on vertex vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573]
2016-05-18 02:07:28,101 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 5D9D05BC43A24444A35D36822985BE74 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:28,102 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:28,102 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:28,103 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:28,103 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:28,103 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 5D9D05BC43A24444A35D36822985BE74 on vertex vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573]
2016-05-18 02:07:28,104 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:28,104 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {5D9D05BC43A24444A35D36822985BE74=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:28,104 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573] parallelism set to 1
2016-05-18 02:07:28,105 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:28,105 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573], numAdditionalOutputs=1
2016-05-18 02:07:28,105 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=8253302BA47247EFAFA64D26BC7553C9, vertex=vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:28,107 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for F758771F2AE14F7A970DF14D4E08C573:8253302BA47247EFAFA64D26BC7553C9 using oldmapred API
2016-05-18 02:07:28,107 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=8253302BA47247EFAFA64D26BC7553C9, vertexName=F758771F2AE14F7A970DF14D4E08C573, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:28,108 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247927_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F758771F2AE14F7A970DF14D4E08C573, vertexId=vertex_1463537247927_0001_1_00, initRequestedTime=1463537248098, initedTime=1463537248105, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:28,108 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573] with distanceFromRoot: 0
2016-05-18 02:07:28,108 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:28,108 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247927_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537247927_0001_1_00, startRequestedTime=1463537248100, startedTime=1463537248108
2016-05-18 02:07:28,108 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247927_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537247927_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:28,108 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:28,109 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in F758771F2AE14F7A970DF14D4E08C573
2016-05-18 02:07:28,109 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573]
2016-05-18 02:07:28,110 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247927_0001_1][Event:TASK_STARTED]: vertexName=F758771F2AE14F7A970DF14D4E08C573, taskId=task_1463537247927_0001_1_00_000000, scheduledTime=1463537248109, launchTime=1463537248109
2016-05-18 02:07:28,110 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573]
2016-05-18 02:07:28,111 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:28,112 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537247927_0001_00_000001, launchTime=1463537248112
2016-05-18 02:07:28,113 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537247927_0001_00_000001
2016-05-18 02:07:28,114 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537247927_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537247927_0001_1_00_000000_0
2016-05-18 02:07:28,115 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537247927_0001_1_00_000000_0] started. Is using containerId: [container_1463537247927_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:28,115 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247927_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F758771F2AE14F7A970DF14D4E08C573, taskAttemptId=attempt_1463537247927_0001_1_00_000000_0, startTime=1463537248114, containerId=container_1463537247927_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:28 Starting to run new task attempt: attempt_1463537247927_0001_1_00_000000_0
2016-05-18 02:07:28,115 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537247927_0001_1_00_000000_0
2016-05-18 02:07:28,116 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:28,117 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:28,117 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D87B42EF1B274C6D9FCB21BCCF6B7380/F80A5FAB7B0649F19B8C5AD73BB7DADA] every.every/(1/1) ...ULTS.ALL_'field2'_'field', VertexName: F758771F2AE14F7A970DF14D4E08C573, VertexParallelism: 1, TaskAttemptID:attempt_1463537247927_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5D9D05BC43A24444A35D36822985BE74, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=8253302BA47247EFAFA64D26BC7553C9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:28,118 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:28,118 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537247927_0001_1_00_000000_0
2016-05-18 02:07:28,119 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537247927_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:28,119 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537247927_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:28,120 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537247927_0001, with dag id: dag_1463537247927_0001_1
2016-05-18 02:07:28,123 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:28,123 INFO input.MRInput (MRInput.java:initialize(440)) - 5D9D05BC43A24444A35D36822985BE74 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:28,124 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 5D9D05BC43A24444A35D36822985BE74
2016-05-18 02:07:28,130 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F758771F2AE14F7A970DF14D4E08C573, ordinal: 0
2016-05-18 02:07:28,130 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F758771F2AE14F7A970DF14D4E08C573, mem on start (mb), free: 199, total: 388, max: 672
2016-05-18 02:07:28,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:28,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:28,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:28,135 INFO output.MROutput (MROutput.java:initialize(421)) - 8253302BA47247EFAFA64D26BC7553C9: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:28,135 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:28,135 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[5D9D05BC43A24444A35D36822985BE74:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [8253302BA47247EFAFA64D26BC7553C9:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:28,136 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:28,136 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:28,136 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:28,136 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[5D9D05BC43A24444A35D36822985BE74:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [8253302BA47247EFAFA64D26BC7553C9:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:28,136 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:28,136 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 5D9D05BC43A24444A35D36822985BE74 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:28,136 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 5D9D05BC43A24444A35D36822985BE74
2016-05-18 02:07:28,136 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:28,137 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:28,137 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:28,137 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537247927_0001_1_00_000000_0
2016-05-18 02:07:28,139 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:28,139 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 5D9D05BC43A24444A35D36822985BE74 initialized RecordReader from event
2016-05-18 02:07:28,140 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:28,141 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 5D9D05BC43A24444A35D36822985BE74
2016-05-18 02:07:28,141 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field'"], id: 8253302BA47247EFAFA64D26BC7553C9
2016-05-18 02:07:28,141 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field'"]
2016-05-18 02:07:28,141 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:28,142 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F758771F2AE14F7A970DF14D4E08C573, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:28,144 INFO output.MROutput (MROutput.java:close(526)) - 8253302BA47247EFAFA64D26BC7553C9 closed
2016-05-18 02:07:28,144 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537247927_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:28,145 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537247927_0001_1_00_000000_0
2016-05-18 02:07:28,145 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F758771F2AE14F7A970DF14D4E08C573, mem on close (mb), free: 197, total: 388, max: 672
2016-05-18 02:07:28,145 INFO output.MROutput (MROutput.java:close(526)) - 8253302BA47247EFAFA64D26BC7553C9 closed
2016-05-18 02:07:28,145 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:28 Completed running task attempt: attempt_1463537247927_0001_1_00_000000_0
2016-05-18 02:07:28,146 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537247927_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:28,146 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537247927_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:28,147 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537247927_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537248142, Process_Duration=3, Process_End_Time=1463537248145, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:28,148 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537247927_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:28,148 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:28,148 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247927_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F758771F2AE14F7A970DF14D4E08C573, taskAttemptId=attempt_1463537247927_0001_1_00_000000_0, creationTime=1463537248110, allocationTime=1463537248112, startTime=1463537248114, finishTime=1463537248148, timeTaken=34, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:28,148 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:28,148 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:28,148 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537247927_0001_1_00_000000_0
2016-05-18 02:07:28,149 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537247927_0001_00_000001
2016-05-18 02:07:28,149 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247927_0001_1][Event:TASK_FINISHED]: vertexName=F758771F2AE14F7A970DF14D4E08C573, taskId=task_1463537247927_0001_1_00_000000, startTime=1463537248114, finishTime=1463537248148, timeTaken=34, status=SUCCEEDED, successfulAttemptID=attempt_1463537247927_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537248142, Process_Duration=3, Process_End_Time=1463537248145, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:28,149 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:28,149 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573]
2016-05-18 02:07:28,150 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247927_0001_1][Event:VERTEX_FINISHED]: vertexName=F758771F2AE14F7A970DF14D4E08C573, vertexId=vertex_1463537247927_0001_1_00, initRequestedTime=1463537248098, initedTime=1463537248105, startRequestedTime=1463537248100, startedTime=1463537248108, finishTime=1463537248149, timeTaken=41, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537248142, Process_Duration=3, Process_End_Time=1463537248145, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537248114, firstTasksToStart=[ task_1463537247927_0001_1_00_000000 ], lastTaskFinishTime=1463537248148, lastTasksToFinish=[ task_1463537247927_0001_1_00_000000 ], minTaskDuration=34, maxTaskDuration=34, avgTaskDuration=34.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537247927_0001_1_00_000000 ], longestDurationTasks=[ task_1463537247927_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:28,150 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:28,151 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247927_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537247927_0001_00_000001, stoppedTime=1463537248150, exitStatus=0
2016-05-18 02:07:28,151 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:28,151 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:28,151 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537247927_0001_1
2016-05-18 02:07:28,151 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247927_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537247927_0001_1
2016-05-18 02:07:28,152 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 8253302BA47247EFAFA64D26BC7553C9 for vertex: vertex_1463537247927_0001_1_00 [F758771F2AE14F7A970DF14D4E08C573], outputName: 8253302BA47247EFAFA64D26BC7553C9
2016-05-18 02:07:28,153 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537247927_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:28,154 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537247927_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:28,159 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537247927_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:28,159 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537247927_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:28,159 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537247927_0001_00_000001, Exiting
2016-05-18 02:07:28,159 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537247927_0001_00_000001
2016-05-18 02:07:28,160 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537247927_0001_00_000001
2016-05-18 02:07:28,160 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537247927_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:28,160 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537247927_0001_00_000001 completed successfully
2016-05-18 02:07:28,162 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:8253302BA47247EFAFA64D26BC7553C9 of vertex/vertexGroup:F758771F2AE14F7A970DF14D4E08C573 isVertexGroupOutput:false
2016-05-18 02:07:28,162 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:28,163 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537247927_0001_1, queueSize=0
2016-05-18 02:07:28,163 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537247927_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537247927_0001_1, startTime=1463537248098, finishTime=1463537248162, timeTaken=64, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=100, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537248142, Process_Duration=3, Process_End_Time=1463537248145, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:28,163 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537247927_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:28,164 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537247927_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:28 Completed Dag: dag_1463537247927_0001_1
2016-05-18 02:07:28,164 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537247927_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:28,164 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537247927_0001_1_post
2016-05-18 02:07:28,164 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:28,164 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:28,164 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:28,164 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[D87B42EF1B274C6D9FCB21BCCF6B7380/F80A5FAB7B0649F19B8C5AD73BB7DADA] every.every/(1/1) ...ULTS.ALL_'field2'_'field', with id=dag_1463537247927_0001_1
2016-05-18 02:07:28,165 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[D87B42EF1B274C6D9FCB21BCCF6B7380/F80A5FAB7B0649F19B8C5AD73BB7DADA] every.every/(1/1) ...ULTS.ALL_'field2'_'field', with id=dag_1463537247927_0001_1
2016-05-18 02:07:28,170 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ULTS.ALL_'field2'_'field', applicationId=application_1463537247927_0001
2016-05-18 02:07:28,172 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:28,172 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:28,172 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:28,172 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:28,172 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:28,172 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:28,174 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:28,174 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:28,174 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:28,174 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:28,175 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:28,175 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:28,175 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:28,176 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:28,178 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/D87B42EF1B274C6D9FCB21BCCF6B7380/F80A5FAB7B0649F19B8C5AD73BB7DADA/.tez/application_1463537247927_0001
2016-05-18 02:07:28,179 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field'/_temporary
2016-05-18 02:07:28,179 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.272
2016-05-18 02:07:28,250 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_'field2'_'field'
2016-05-18 02:07:28,273 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:28,274 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:28,311 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-18 02:07:28,320 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-18 02:07:28,320 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:28,321 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:28,321 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:28,342 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:28,343 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_RESULTS
2016-05-18 02:07:28,352 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:28,352 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:28,352 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_RESULTS"]
2016-05-18 02:07:28,352 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:28,353 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:28,354 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:28,354 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ULTS.ALL_'field2'_RESULTS
2016-05-18 02:07:28,355 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:28,370 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:28,375 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:28,375 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:28,375 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:28,384 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/052515D35B6A41D785BDC8E393341773/F21E922DFE9B4A168CA4F8AA363CD3D6/.tez/application_1463537248375_0001 doesn't exist and is created
2016-05-18 02:07:28,404 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:28,405 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:28,405 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/052515D35B6A41D785BDC8E393341773/F21E922DFE9B4A168CA4F8AA363CD3D6/.tez/application_1463537248375_0001_wd
2016-05-18 02:07:28,420 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537248375_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:28,420 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:28,420 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:28,422 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:28,422 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:28,423 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:28,423 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:28,424 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:28,425 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:28,425 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:28,425 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:28,425 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:28,426 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:28,426 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:28,426 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:28,426 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:28,426 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:28,426 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:28,427 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:28,427 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:28,429 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:28,430 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:28,430 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:28,430 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/052515D35B6A41D785BDC8E393341773/F21E922DFE9B4A168CA4F8AA363CD3D6/.tez/application_1463537248375_0001_wd/localmode-log-dir
2016-05-18 02:07:28,430 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/052515D35B6A41D785BDC8E393341773/F21E922DFE9B4A168CA4F8AA363CD3D6/.tez/application_1463537248375_0001_wd/localmode-log-dir/history.txt.appattempt_1463537248375_0001_000000, maxErrors=10
2016-05-18 02:07:28,431 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/052515D35B6A41D785BDC8E393341773/F21E922DFE9B4A168CA4F8AA363CD3D6/.tez/application_1463537248375_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:28,431 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537248375_0001_000000, appSubmitTime=1463537248420, launchTime=1463537248420
2016-05-18 02:07:28,433 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:28,439 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:54165
2016-05-18 02:07:28,442 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537248375
2016-05-18 02:07:28,444 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537248375_0001_000000, startTime=1463537248444
2016-05-18 02:07:28,444 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:28,505 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:28,505 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:28,505 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ULTS.ALL_'field2'_RESULTS, applicationId=application_1463537248375_0001, dagName=[052515D35B6A41D785BDC8E393341773/F21E922DFE9B4A168CA4F8AA363CD3D6] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS
2016-05-18 02:07:28,521 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [052515D35B6A41D785BDC8E393341773/F21E922DFE9B4A168CA4F8AA363CD3D6] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS
2016-05-18 02:07:28,521 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 02:07:28,522 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 02:07:28,522 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:28,523 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537248375_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/052515D35B6A41D785BDC8E393341773/F21E922DFE9B4A168CA4F8AA363CD3D6/.tez/application_1463537248375_0001_wd/localmode-log-dir/dag_1463537248375_0001_1.dot
2016-05-18 02:07:28,523 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/052515D35B6A41D785BDC8E393341773/F21E922DFE9B4A168CA4F8AA363CD3D6/.tez/application_1463537248375_0001_wd/localmode-log-dir/dag_1463537248375_0001_1-tez-dag.pb.txt
2016-05-18 02:07:28,527 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537248375_0001_1
2016-05-18 02:07:28,527 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:28,527 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [052515D35B6A41D785BDC8E393341773/F21E922DFE9B4A168CA4F8AA363CD3D6] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS
2016-05-18 02:07:28 Running Dag: dag_1463537248375_0001_1
2016-05-18 02:07:28,542 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248375_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537248375_0001_1, submitTime=1463537248522
2016-05-18 02:07:28,543 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/052515D35B6A41D785BDC8E393341773/F21E922DFE9B4A168CA4F8AA363CD3D6/.tez/application_1463537248375_0001/tez-conf.pb" } size: 40947 timestamp: 1463537248000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:28,543 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307]
2016-05-18 02:07:28,543 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307]
2016-05-18 02:07:28,544 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:28,544 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:28,545 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248375_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537248375_0001_1, initTime=1463537248544
2016-05-18 02:07:28,545 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537248375_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:28,545 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:28,546 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248375_0001_1][Event:DAG_STARTED]: dagID=dag_1463537248375_0001_1, startTime=1463537248546
2016-05-18 02:07:28,546 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537248375_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:28,546 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: B3FADC60A3B14658A1782FAB389B9307 : {D37234C280CD4D988B28A3F646A31E1B={InputName=D37234C280CD4D988B28A3F646A31E1B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:28,546 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: D37234C280CD4D988B28A3F646A31E1B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:28,546 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307]
2016-05-18 02:07:28,546 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307]
2016-05-18 02:07:28,547 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307]
2016-05-18 02:07:28,547 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307]
2016-05-18 02:07:28,547 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ULTS.ALL_'field2'_RESULTS, applicationId=application_1463537248375_0001, dagName=[052515D35B6A41D785BDC8E393341773/F21E922DFE9B4A168CA4F8AA363CD3D6] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS
2016-05-18 02:07:28,548 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:28,548 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: D37234C280CD4D988B28A3F646A31E1B on vertex vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307]
2016-05-18 02:07:28,550 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input D37234C280CD4D988B28A3F646A31E1B asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:28,551 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:28,551 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:28,551 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:28,551 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:28,551 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: D37234C280CD4D988B28A3F646A31E1B on vertex vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307]
2016-05-18 02:07:28,553 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:28,554 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {D37234C280CD4D988B28A3F646A31E1B=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:28,555 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307] parallelism set to 1
2016-05-18 02:07:28,555 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:28,555 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307], numAdditionalOutputs=1
2016-05-18 02:07:28,555 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=039FB3621D414621AF1D99C3AAECB2BC, vertex=vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:28,567 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for B3FADC60A3B14658A1782FAB389B9307:039FB3621D414621AF1D99C3AAECB2BC using oldmapred API
2016-05-18 02:07:28,568 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=039FB3621D414621AF1D99C3AAECB2BC, vertexName=B3FADC60A3B14658A1782FAB389B9307, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:28,568 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248375_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B3FADC60A3B14658A1782FAB389B9307, vertexId=vertex_1463537248375_0001_1_00, initRequestedTime=1463537248546, initedTime=1463537248555, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:28,568 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307] with distanceFromRoot: 0
2016-05-18 02:07:28,568 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:28,569 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248375_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537248375_0001_1_00, startRequestedTime=1463537248548, startedTime=1463537248569
2016-05-18 02:07:28,569 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248375_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537248375_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:28,570 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:28,570 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in B3FADC60A3B14658A1782FAB389B9307
2016-05-18 02:07:28,570 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307]
2016-05-18 02:07:28,571 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248375_0001_1][Event:TASK_STARTED]: vertexName=B3FADC60A3B14658A1782FAB389B9307, taskId=task_1463537248375_0001_1_00_000000, scheduledTime=1463537248570, launchTime=1463537248570
2016-05-18 02:07:28,571 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307]
2016-05-18 02:07:28,572 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:28,574 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537248375_0001_00_000001, launchTime=1463537248574
2016-05-18 02:07:28,575 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537248375_0001_00_000001
2016-05-18 02:07:28,576 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537248375_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537248375_0001_1_00_000000_0
2016-05-18 02:07:28,576 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537248375_0001_1_00_000000_0] started. Is using containerId: [container_1463537248375_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:28,576 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248375_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B3FADC60A3B14658A1782FAB389B9307, taskAttemptId=attempt_1463537248375_0001_1_00_000000_0, startTime=1463537248576, containerId=container_1463537248375_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:28 Starting to run new task attempt: attempt_1463537248375_0001_1_00_000000_0
2016-05-18 02:07:28,578 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537248375_0001_1_00_000000_0
2016-05-18 02:07:28,578 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:28,579 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:28,579 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [052515D35B6A41D785BDC8E393341773/F21E922DFE9B4A168CA4F8AA363CD3D6] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS, VertexName: B3FADC60A3B14658A1782FAB389B9307, VertexParallelism: 1, TaskAttemptID:attempt_1463537248375_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D37234C280CD4D988B28A3F646A31E1B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=039FB3621D414621AF1D99C3AAECB2BC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:28,580 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:28,580 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537248375_0001, with dag id: dag_1463537248375_0001_1
2016-05-18 02:07:28,581 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537248375_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:28,582 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537248375_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:28,581 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537248375_0001_1_00_000000_0
2016-05-18 02:07:28,586 INFO input.MRInput (MRInput.java:initialize(440)) - D37234C280CD4D988B28A3F646A31E1B using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:28,586 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: D37234C280CD4D988B28A3F646A31E1B
2016-05-18 02:07:28,586 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:28,593 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B3FADC60A3B14658A1782FAB389B9307, ordinal: 0
2016-05-18 02:07:28,593 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B3FADC60A3B14658A1782FAB389B9307, mem on start (mb), free: 313, total: 388, max: 672
2016-05-18 02:07:28,593 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:28,593 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:28,593 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:28,597 INFO output.MROutput (MROutput.java:initialize(421)) - 039FB3621D414621AF1D99C3AAECB2BC: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:28,597 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:28,597 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[D37234C280CD4D988B28A3F646A31E1B:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [039FB3621D414621AF1D99C3AAECB2BC:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:28,598 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:28,598 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:28,598 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:28,598 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[D37234C280CD4D988B28A3F646A31E1B:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [039FB3621D414621AF1D99C3AAECB2BC:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:28,598 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:28,598 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: D37234C280CD4D988B28A3F646A31E1B
2016-05-18 02:07:28,598 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: D37234C280CD4D988B28A3F646A31E1B being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:28,598 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:28,599 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:28,599 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:28,599 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537248375_0001_1_00_000000_0
2016-05-18 02:07:28,601 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:28,601 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - D37234C280CD4D988B28A3F646A31E1B initialized RecordReader from event
2016-05-18 02:07:28,602 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:28,602 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: D37234C280CD4D988B28A3F646A31E1B
2016-05-18 02:07:28,603 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_RESULTS"], id: 039FB3621D414621AF1D99C3AAECB2BC
2016-05-18 02:07:28,603 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_RESULTS"]
2016-05-18 02:07:28,603 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:28,603 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B3FADC60A3B14658A1782FAB389B9307, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:28,605 INFO output.MROutput (MROutput.java:close(526)) - 039FB3621D414621AF1D99C3AAECB2BC closed
2016-05-18 02:07:28,606 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537248375_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:28,606 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537248375_0001_1_00_000000_0
2016-05-18 02:07:28,606 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B3FADC60A3B14658A1782FAB389B9307, mem on close (mb), free: 311, total: 388, max: 672
2016-05-18 02:07:28,607 INFO output.MROutput (MROutput.java:close(526)) - 039FB3621D414621AF1D99C3AAECB2BC closed
2016-05-18 02:07:28,607 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:28 Completed running task attempt: attempt_1463537248375_0001_1_00_000000_0
2016-05-18 02:07:28,608 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537248375_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:28,609 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537248375_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:28,609 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537248375_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537248603, Process_Duration=3, Process_End_Time=1463537248606, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:28,610 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537248375_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:28,610 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248375_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B3FADC60A3B14658A1782FAB389B9307, taskAttemptId=attempt_1463537248375_0001_1_00_000000_0, creationTime=1463537248571, allocationTime=1463537248574, startTime=1463537248576, finishTime=1463537248610, timeTaken=34, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:28,610 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:28,610 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:28,611 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:28,610 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537248375_0001_1_00_000000_0
2016-05-18 02:07:28,611 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537248375_0001_00_000001
2016-05-18 02:07:28,611 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248375_0001_1][Event:TASK_FINISHED]: vertexName=B3FADC60A3B14658A1782FAB389B9307, taskId=task_1463537248375_0001_1_00_000000, startTime=1463537248576, finishTime=1463537248611, timeTaken=35, status=SUCCEEDED, successfulAttemptID=attempt_1463537248375_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537248603, Process_Duration=3, Process_End_Time=1463537248606, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:28,612 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:28,612 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307]
2016-05-18 02:07:28,613 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248375_0001_1][Event:VERTEX_FINISHED]: vertexName=B3FADC60A3B14658A1782FAB389B9307, vertexId=vertex_1463537248375_0001_1_00, initRequestedTime=1463537248546, initedTime=1463537248555, startRequestedTime=1463537248548, startedTime=1463537248569, finishTime=1463537248612, timeTaken=43, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537248603, Process_Duration=3, Process_End_Time=1463537248606, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537248576, firstTasksToStart=[ task_1463537248375_0001_1_00_000000 ], lastTaskFinishTime=1463537248610, lastTasksToFinish=[ task_1463537248375_0001_1_00_000000 ], minTaskDuration=34, maxTaskDuration=34, avgTaskDuration=34.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537248375_0001_1_00_000000 ], longestDurationTasks=[ task_1463537248375_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:28,613 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:28,613 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248375_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537248375_0001_00_000001, stoppedTime=1463537248613, exitStatus=0
2016-05-18 02:07:28,613 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:28,613 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:28,614 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537248375_0001_1
2016-05-18 02:07:28,614 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248375_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537248375_0001_1
2016-05-18 02:07:28,614 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 039FB3621D414621AF1D99C3AAECB2BC for vertex: vertex_1463537248375_0001_1_00 [B3FADC60A3B14658A1782FAB389B9307], outputName: 039FB3621D414621AF1D99C3AAECB2BC
2016-05-18 02:07:28,616 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537248375_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:28,616 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537248375_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:28,622 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537248375_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:28,622 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537248375_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:28,622 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537248375_0001_00_000001, Exiting
2016-05-18 02:07:28,622 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537248375_0001_00_000001
2016-05-18 02:07:28,623 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537248375_0001_00_000001
2016-05-18 02:07:28,623 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537248375_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:28,624 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537248375_0001_00_000001 completed successfully
2016-05-18 02:07:28,628 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:039FB3621D414621AF1D99C3AAECB2BC of vertex/vertexGroup:B3FADC60A3B14658A1782FAB389B9307 isVertexGroupOutput:false
2016-05-18 02:07:28,628 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:28,630 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537248375_0001_1, queueSize=0
2016-05-18 02:07:28,630 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248375_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537248375_0001_1, startTime=1463537248545, finishTime=1463537248628, timeTaken=83, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=170, AM_GC_TIME_MILLIS=10, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537248603, Process_Duration=3, Process_End_Time=1463537248606, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:28,630 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537248375_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:28,631 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537248375_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:28 Completed Dag: dag_1463537248375_0001_1
2016-05-18 02:07:28,631 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537248375_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:28,631 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537248375_0001_1_post
2016-05-18 02:07:28,631 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:28,631 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:28,631 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:28,632 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[052515D35B6A41D785BDC8E393341773/F21E922DFE9B4A168CA4F8AA363CD3D6] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS, with id=dag_1463537248375_0001_1
2016-05-18 02:07:28,632 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[052515D35B6A41D785BDC8E393341773/F21E922DFE9B4A168CA4F8AA363CD3D6] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS, with id=dag_1463537248375_0001_1
2016-05-18 02:07:28,639 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ULTS.ALL_'field2'_RESULTS, applicationId=application_1463537248375_0001
2016-05-18 02:07:28,641 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:28,641 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:28,641 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:28,641 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:28,641 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:28,642 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:28,642 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:28,643 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:28,643 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:28,643 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:28,644 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:28,644 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:28,644 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:28,645 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:28,648 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/052515D35B6A41D785BDC8E393341773/F21E922DFE9B4A168CA4F8AA363CD3D6/.tez/application_1463537248375_0001
2016-05-18 02:07:28,649 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_RESULTS/_temporary
2016-05-18 02:07:28,649 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.297
2016-05-18 02:07:28,722 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_'field2'_RESULTS
2016-05-18 02:07:28,746 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:28,747 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:28,791 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-18 02:07:28,799 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-18 02:07:28,799 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:28,800 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:28,800 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:28,822 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:28,823 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_ALL
2016-05-18 02:07:28,836 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:28,836 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:28,836 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_ALL"]
2016-05-18 02:07:28,837 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:28,837 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:28,837 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:28,838 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ..._RESULTS.ALL_'field2'_ALL
2016-05-18 02:07:28,838 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:28,858 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:28,863 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:28,864 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:28,864 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:28,874 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9D86ADB0B4734E01B71CE6C4ACB9E8DB/E3C43E9F92204650A1A6851E321C4225/.tez/application_1463537248864_0001 doesn't exist and is created
2016-05-18 02:07:28,897 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:28,897 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:28,900 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9D86ADB0B4734E01B71CE6C4ACB9E8DB/E3C43E9F92204650A1A6851E321C4225/.tez/application_1463537248864_0001_wd
2016-05-18 02:07:28,918 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537248864_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:28,919 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:28,919 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:28,921 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:28,922 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:28,922 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:28,922 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:28,923 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:28,925 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:28,925 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:28,926 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:28,926 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:28,926 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:28,926 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:28,926 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:28,928 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:28,928 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:28,928 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:28,928 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:28,928 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:28,930 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:28,931 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:28,931 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:28,931 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9D86ADB0B4734E01B71CE6C4ACB9E8DB/E3C43E9F92204650A1A6851E321C4225/.tez/application_1463537248864_0001_wd/localmode-log-dir
2016-05-18 02:07:28,931 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9D86ADB0B4734E01B71CE6C4ACB9E8DB/E3C43E9F92204650A1A6851E321C4225/.tez/application_1463537248864_0001_wd/localmode-log-dir/history.txt.appattempt_1463537248864_0001_000000, maxErrors=10
2016-05-18 02:07:28,932 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9D86ADB0B4734E01B71CE6C4ACB9E8DB/E3C43E9F92204650A1A6851E321C4225/.tez/application_1463537248864_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:28,932 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537248864_0001_000000, appSubmitTime=1463537248918, launchTime=1463537248918
2016-05-18 02:07:28,935 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:28,939 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:35339
2016-05-18 02:07:28,943 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537248864
2016-05-18 02:07:28,946 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537248864_0001_000000, startTime=1463537248946
2016-05-18 02:07:28,947 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:28,998 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:28,998 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:28,998 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ..._RESULTS.ALL_'field2'_ALL, applicationId=application_1463537248864_0001, dagName=[9D86ADB0B4734E01B71CE6C4ACB9E8DB/E3C43E9F92204650A1A6851E321C4225] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL
2016-05-18 02:07:29,014 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [9D86ADB0B4734E01B71CE6C4ACB9E8DB/E3C43E9F92204650A1A6851E321C4225] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL
2016-05-18 02:07:29,015 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:29,016 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537248864_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9D86ADB0B4734E01B71CE6C4ACB9E8DB/E3C43E9F92204650A1A6851E321C4225/.tez/application_1463537248864_0001_wd/localmode-log-dir/dag_1463537248864_0001_1.dot
2016-05-18 02:07:29,016 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9D86ADB0B4734E01B71CE6C4ACB9E8DB/E3C43E9F92204650A1A6851E321C4225/.tez/application_1463537248864_0001_wd/localmode-log-dir/dag_1463537248864_0001_1-tez-dag.pb.txt
2016-05-18 02:07:29,020 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537248864_0001_1
2016-05-18 02:07:29,020 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:29,020 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [9D86ADB0B4734E01B71CE6C4ACB9E8DB/E3C43E9F92204650A1A6851E321C4225] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL
2016-05-18 02:07:29 Running Dag: dag_1463537248864_0001_1
2016-05-18 02:07:29,035 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248864_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537248864_0001_1, submitTime=1463537249015
2016-05-18 02:07:29,036 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9D86ADB0B4734E01B71CE6C4ACB9E8DB/E3C43E9F92204650A1A6851E321C4225/.tez/application_1463537248864_0001/tez-conf.pb" } size: 40947 timestamp: 1463537248000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:29,036 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50]
2016-05-18 02:07:29,037 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50]
2016-05-18 02:07:29,037 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:29,037 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:29,037 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248864_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537248864_0001_1, initTime=1463537249037
2016-05-18 02:07:29,038 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537248864_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:29,038 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:29,038 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248864_0001_1][Event:DAG_STARTED]: dagID=dag_1463537248864_0001_1, startTime=1463537249038
2016-05-18 02:07:29,039 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537248864_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:29,039 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: D0062B6B6C3A434BA610EF5EAC48BA50 : {F80277CB5AE143F8A7DD183242D155CF={InputName=F80277CB5AE143F8A7DD183242D155CF}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:29,039 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: F80277CB5AE143F8A7DD183242D155CF, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:29,039 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50]
2016-05-18 02:07:29,039 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ..._RESULTS.ALL_'field2'_ALL, applicationId=application_1463537248864_0001, dagName=[9D86ADB0B4734E01B71CE6C4ACB9E8DB/E3C43E9F92204650A1A6851E321C4225] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL
2016-05-18 02:07:29,039 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50]
2016-05-18 02:07:29,039 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50]
2016-05-18 02:07:29,039 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50]
2016-05-18 02:07:29,040 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:29,040 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: F80277CB5AE143F8A7DD183242D155CF on vertex vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50]
2016-05-18 02:07:29,042 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input F80277CB5AE143F8A7DD183242D155CF asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:29,043 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:29,043 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:29,043 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:29,043 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:29,044 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: F80277CB5AE143F8A7DD183242D155CF on vertex vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50]
2016-05-18 02:07:29,047 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:29,047 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {F80277CB5AE143F8A7DD183242D155CF=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:29,049 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50] parallelism set to 1
2016-05-18 02:07:29,050 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:29,050 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50], numAdditionalOutputs=1
2016-05-18 02:07:29,050 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=4711AC571AAC4F87A7333427C1AFB2FB, vertex=vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:29,052 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D0062B6B6C3A434BA610EF5EAC48BA50:4711AC571AAC4F87A7333427C1AFB2FB using oldmapred API
2016-05-18 02:07:29,052 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=4711AC571AAC4F87A7333427C1AFB2FB, vertexName=D0062B6B6C3A434BA610EF5EAC48BA50, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:29,053 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248864_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D0062B6B6C3A434BA610EF5EAC48BA50, vertexId=vertex_1463537248864_0001_1_00, initRequestedTime=1463537249039, initedTime=1463537249050, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:29,053 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50] with distanceFromRoot: 0
2016-05-18 02:07:29,053 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:29,055 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 02:07:29,056 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 02:07:29,056 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248864_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537248864_0001_1_00, startRequestedTime=1463537249040, startedTime=1463537249056
2016-05-18 02:07:29,057 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248864_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537248864_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:29,057 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:29,057 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in D0062B6B6C3A434BA610EF5EAC48BA50
2016-05-18 02:07:29,058 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50]
2016-05-18 02:07:29,059 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248864_0001_1][Event:TASK_STARTED]: vertexName=D0062B6B6C3A434BA610EF5EAC48BA50, taskId=task_1463537248864_0001_1_00_000000, scheduledTime=1463537249058, launchTime=1463537249058
2016-05-18 02:07:29,059 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50]
2016-05-18 02:07:29,060 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:29,061 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537248864_0001_00_000001, launchTime=1463537249061
2016-05-18 02:07:29,062 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537248864_0001_00_000001
2016-05-18 02:07:29,063 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537248864_0001_1_00_000000_0] started. Is using containerId: [container_1463537248864_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:29,063 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248864_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D0062B6B6C3A434BA610EF5EAC48BA50, taskAttemptId=attempt_1463537248864_0001_1_00_000000_0, startTime=1463537249062, containerId=container_1463537248864_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:29,063 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537248864_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537248864_0001_1_00_000000_0
2016-05-18 02:07:29 Starting to run new task attempt: attempt_1463537248864_0001_1_00_000000_0
2016-05-18 02:07:29,064 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537248864_0001_1_00_000000_0
2016-05-18 02:07:29,064 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:29,065 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:29,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [9D86ADB0B4734E01B71CE6C4ACB9E8DB/E3C43E9F92204650A1A6851E321C4225] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL, VertexName: D0062B6B6C3A434BA610EF5EAC48BA50, VertexParallelism: 1, TaskAttemptID:attempt_1463537248864_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F80277CB5AE143F8A7DD183242D155CF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=4711AC571AAC4F87A7333427C1AFB2FB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:29,065 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:29,066 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537248864_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:29,066 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537248864_0001_1_00_000000_0
2016-05-18 02:07:29,066 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537248864_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:29,067 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537248864_0001, with dag id: dag_1463537248864_0001_1
2016-05-18 02:07:29,071 INFO input.MRInput (MRInput.java:initialize(440)) - F80277CB5AE143F8A7DD183242D155CF using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:29,071 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: F80277CB5AE143F8A7DD183242D155CF
2016-05-18 02:07:29,074 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:29,082 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D0062B6B6C3A434BA610EF5EAC48BA50, ordinal: 0
2016-05-18 02:07:29,082 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D0062B6B6C3A434BA610EF5EAC48BA50, mem on start (mb), free: 218, total: 388, max: 672
2016-05-18 02:07:29,082 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:29,082 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:29,082 INFO output.MROutput (MROutput.java:initialize(421)) - 4711AC571AAC4F87A7333427C1AFB2FB: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:29,082 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:29,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:29,083 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[F80277CB5AE143F8A7DD183242D155CF:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [4711AC571AAC4F87A7333427C1AFB2FB:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:29,083 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:29,083 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:29,084 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:29,084 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[F80277CB5AE143F8A7DD183242D155CF:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [4711AC571AAC4F87A7333427C1AFB2FB:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:29,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:29,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: F80277CB5AE143F8A7DD183242D155CF being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:29,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: F80277CB5AE143F8A7DD183242D155CF
2016-05-18 02:07:29,085 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:29,085 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:29,085 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:29,086 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537248864_0001_1_00_000000_0
2016-05-18 02:07:29,089 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:29,089 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:29,090 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - F80277CB5AE143F8A7DD183242D155CF initialized RecordReader from event
2016-05-18 02:07:29,090 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: F80277CB5AE143F8A7DD183242D155CF
2016-05-18 02:07:29,091 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_ALL"], id: 4711AC571AAC4F87A7333427C1AFB2FB
2016-05-18 02:07:29,091 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_ALL"]
2016-05-18 02:07:29,091 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:29,091 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D0062B6B6C3A434BA610EF5EAC48BA50, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:29,094 INFO output.MROutput (MROutput.java:close(526)) - 4711AC571AAC4F87A7333427C1AFB2FB closed
2016-05-18 02:07:29,095 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537248864_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:29,095 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537248864_0001_1_00_000000_0
2016-05-18 02:07:29,095 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D0062B6B6C3A434BA610EF5EAC48BA50, mem on close (mb), free: 216, total: 388, max: 672
2016-05-18 02:07:29,095 INFO output.MROutput (MROutput.java:close(526)) - 4711AC571AAC4F87A7333427C1AFB2FB closed
2016-05-18 02:07:29,096 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:29 Completed running task attempt: attempt_1463537248864_0001_1_00_000000_0
2016-05-18 02:07:29,097 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537248864_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:29,097 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537248864_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:29,097 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537248864_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537249092, Process_Duration=3, Process_End_Time=1463537249095, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:29,099 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537248864_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:29,099 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:29,099 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248864_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D0062B6B6C3A434BA610EF5EAC48BA50, taskAttemptId=attempt_1463537248864_0001_1_00_000000_0, creationTime=1463537249059, allocationTime=1463537249061, startTime=1463537249062, finishTime=1463537249099, timeTaken=37, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:29,099 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:29,099 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:29,099 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537248864_0001_1_00_000000_0
2016-05-18 02:07:29,100 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248864_0001_1][Event:TASK_FINISHED]: vertexName=D0062B6B6C3A434BA610EF5EAC48BA50, taskId=task_1463537248864_0001_1_00_000000, startTime=1463537249062, finishTime=1463537249099, timeTaken=37, status=SUCCEEDED, successfulAttemptID=attempt_1463537248864_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537249092, Process_Duration=3, Process_End_Time=1463537249095, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:29,100 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:29,100 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50]
2016-05-18 02:07:29,101 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537248864_0001_00_000001
2016-05-18 02:07:29,101 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248864_0001_1][Event:VERTEX_FINISHED]: vertexName=D0062B6B6C3A434BA610EF5EAC48BA50, vertexId=vertex_1463537248864_0001_1_00, initRequestedTime=1463537249039, initedTime=1463537249050, startRequestedTime=1463537249040, startedTime=1463537249056, finishTime=1463537249100, timeTaken=44, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537249092, Process_Duration=3, Process_End_Time=1463537249095, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537249062, firstTasksToStart=[ task_1463537248864_0001_1_00_000000 ], lastTaskFinishTime=1463537249099, lastTasksToFinish=[ task_1463537248864_0001_1_00_000000 ], minTaskDuration=37, maxTaskDuration=37, avgTaskDuration=37.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537248864_0001_1_00_000000 ], longestDurationTasks=[ task_1463537248864_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:29,102 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:29,102 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:29,102 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:29,102 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537248864_0001_1
2016-05-18 02:07:29,103 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248864_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537248864_0001_1
2016-05-18 02:07:29,104 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 4711AC571AAC4F87A7333427C1AFB2FB for vertex: vertex_1463537248864_0001_1_00 [D0062B6B6C3A434BA610EF5EAC48BA50], outputName: 4711AC571AAC4F87A7333427C1AFB2FB
2016-05-18 02:07:29,107 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537248864_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:29,107 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248864_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537248864_0001_00_000001, stoppedTime=1463537249107, exitStatus=0
2016-05-18 02:07:29,109 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537248864_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:29,109 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537248864_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:29,110 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537248864_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:29,110 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537248864_0001_00_000001, Exiting
2016-05-18 02:07:29,110 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537248864_0001_00_000001
2016-05-18 02:07:29,110 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537248864_0001_00_000001
2016-05-18 02:07:29,113 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537248864_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:29,113 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537248864_0001_00_000001 completed successfully
2016-05-18 02:07:29,121 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:4711AC571AAC4F87A7333427C1AFB2FB of vertex/vertexGroup:D0062B6B6C3A434BA610EF5EAC48BA50 isVertexGroupOutput:false
2016-05-18 02:07:29,121 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:29,126 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537248864_0001_1, queueSize=0
2016-05-18 02:07:29,127 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537248864_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537248864_0001_1, startTime=1463537249038, finishTime=1463537249121, timeTaken=83, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=190, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537249092, Process_Duration=3, Process_End_Time=1463537249095, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:29,127 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537248864_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:29,128 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537248864_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:29 Completed Dag: dag_1463537248864_0001_1
2016-05-18 02:07:29,129 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537248864_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:29,130 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537248864_0001_1_post
2016-05-18 02:07:29,130 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:29,132 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:29,134 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:29,134 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[9D86ADB0B4734E01B71CE6C4ACB9E8DB/E3C43E9F92204650A1A6851E321C4225] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL, with id=dag_1463537248864_0001_1
2016-05-18 02:07:29,135 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[9D86ADB0B4734E01B71CE6C4ACB9E8DB/E3C43E9F92204650A1A6851E321C4225] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL, with id=dag_1463537248864_0001_1
2016-05-18 02:07:29,136 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ..._RESULTS.ALL_'field2'_ALL, applicationId=application_1463537248864_0001
2016-05-18 02:07:29,138 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:29,138 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:29,139 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:29,139 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:29,139 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:29,139 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:29,139 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:29,140 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:29,140 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:29,140 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:29,141 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:29,141 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:29,141 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:29,142 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:29,146 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/9D86ADB0B4734E01B71CE6C4ACB9E8DB/E3C43E9F92204650A1A6851E321C4225/.tez/application_1463537248864_0001
2016-05-18 02:07:29,147 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_ALL/_temporary
2016-05-18 02:07:29,147 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.311
2016-05-18 02:07:29,225 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_'field2'_ALL
2016-05-18 02:07:29,248 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:29,249 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:29,287 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-18 02:07:29,290 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-18 02:07:29,290 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:29,290 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 02:07:29,291 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 02:07:29,306 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt
2016-05-18 02:07:29,306 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_UNKNOWN_-1
2016-05-18 02:07:29,316 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-18 02:07:29,316 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:29,317 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_UNKNOWN_-1"]
2016-05-18 02:07:29,317 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-18 02:07:29,318 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-18 02:07:29,318 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-18 02:07:29,319 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...d'_RESULTS.ALL_UNKNOWN_-1
2016-05-18 02:07:29,319 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 02:07:29,332 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-18 02:07:29,337 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:29,337 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 02:07:29,337 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 02:07:29,347 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8199665CE6554C9FA27982B7D247FF9F/F61222D7BBFC4A92810F50B0E11EAE64/.tez/application_1463537249337_0001 doesn't exist and is created
2016-05-18 02:07:29,369 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 02:07:29,369 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 02:07:29,370 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8199665CE6554C9FA27982B7D247FF9F/F61222D7BBFC4A92810F50B0E11EAE64/.tez/application_1463537249337_0001_wd
2016-05-18 02:07:29,385 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463537249337_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 02:07:29,386 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:29,386 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 02:07:29,387 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 02:07:29,388 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 02:07:29,388 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 02:07:29,388 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 02:07:29,389 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 02:07:29,390 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 02:07:29,390 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 02:07:29,390 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 02:07:29,391 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 02:07:29,391 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 02:07:29,391 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 02:07:29,391 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 02:07:29,392 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 02:07:29,392 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:29,392 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 02:07:29,392 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 02:07:29,393 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 02:07:29,394 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 02:07:29,395 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 02:07:29,395 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 02:07:29,395 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8199665CE6554C9FA27982B7D247FF9F/F61222D7BBFC4A92810F50B0E11EAE64/.tez/application_1463537249337_0001_wd/localmode-log-dir
2016-05-18 02:07:29,396 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8199665CE6554C9FA27982B7D247FF9F/F61222D7BBFC4A92810F50B0E11EAE64/.tez/application_1463537249337_0001_wd/localmode-log-dir/history.txt.appattempt_1463537249337_0001_000000, maxErrors=10
2016-05-18 02:07:29,396 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8199665CE6554C9FA27982B7D247FF9F/F61222D7BBFC4A92810F50B0E11EAE64/.tez/application_1463537249337_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 02:07:29,396 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463537249337_0001_000000, appSubmitTime=1463537249385, launchTime=1463537249385
2016-05-18 02:07:29,398 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 02:07:29,402 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:55778
2016-05-18 02:07:29,402 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463537249337
2016-05-18 02:07:29,405 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463537249337_0001_000000, startTime=1463537249405
2016-05-18 02:07:29,405 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 02:07:29,469 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 02:07:29,469 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 02:07:29,469 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1, applicationId=application_1463537249337_0001, dagName=[8199665CE6554C9FA27982B7D247FF9F/F61222D7BBFC4A92810F50B0E11EAE64] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1
2016-05-18 02:07:29,485 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [8199665CE6554C9FA27982B7D247FF9F/F61222D7BBFC4A92810F50B0E11EAE64] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1
2016-05-18 02:07:29,485 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 02:07:29,486 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463537249337_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8199665CE6554C9FA27982B7D247FF9F/F61222D7BBFC4A92810F50B0E11EAE64/.tez/application_1463537249337_0001_wd/localmode-log-dir/dag_1463537249337_0001_1.dot
2016-05-18 02:07:29,486 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8199665CE6554C9FA27982B7D247FF9F/F61222D7BBFC4A92810F50B0E11EAE64/.tez/application_1463537249337_0001_wd/localmode-log-dir/dag_1463537249337_0001_1-tez-dag.pb.txt
2016-05-18 02:07:29,490 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537249337_0001_1
2016-05-18 02:07:29,490 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:29,490 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [8199665CE6554C9FA27982B7D247FF9F/F61222D7BBFC4A92810F50B0E11EAE64] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1
2016-05-18 02:07:29 Running Dag: dag_1463537249337_0001_1
2016-05-18 02:07:29,506 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537249337_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463537249337_0001_1, submitTime=1463537249485
2016-05-18 02:07:29,507 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8199665CE6554C9FA27982B7D247FF9F/F61222D7BBFC4A92810F50B0E11EAE64/.tez/application_1463537249337_0001/tez-conf.pb" } size: 40947 timestamp: 1463537249000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71], Default Resources=<memory:1024, vCores:1>
2016-05-18 02:07:29,507 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71]
2016-05-18 02:07:29,507 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71]
2016-05-18 02:07:29,508 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 02:07:29,508 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 02:07:29,509 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537249337_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463537249337_0001_1, initTime=1463537249508
2016-05-18 02:07:29,510 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537249337_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 02:07:29,510 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 02:07:29,510 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537249337_0001_1][Event:DAG_STARTED]: dagID=dag_1463537249337_0001_1, startTime=1463537249510
2016-05-18 02:07:29,510 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537249337_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 02:07:29,510 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: A0E40D5704B34F4BAD9BC9033C3A7F71 : {E85B1C0B0DF842188C1C53D849C902C2={InputName=E85B1C0B0DF842188C1C53D849C902C2}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 02:07:29,511 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: E85B1C0B0DF842188C1C53D849C902C2, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 02:07:29,511 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71]
2016-05-18 02:07:29,511 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71]
2016-05-18 02:07:29,511 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71]
2016-05-18 02:07:29,511 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71]
2016-05-18 02:07:29,512 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1, applicationId=application_1463537249337_0001, dagName=[8199665CE6554C9FA27982B7D247FF9F/F61222D7BBFC4A92810F50B0E11EAE64] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1
2016-05-18 02:07:29,512 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 02:07:29,513 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: E85B1C0B0DF842188C1C53D849C902C2 on vertex vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71]
2016-05-18 02:07:29,515 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input E85B1C0B0DF842188C1C53D849C902C2 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 02:07:29,517 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 02:07:29,517 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 51 Min splitLength: 52428800 New desired splits: 1 Total length: 51 Original splits: 2
2016-05-18 02:07:29,517 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 51 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 51 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 02:07:29,517 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 02:07:29,518 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: E85B1C0B0DF842188C1C53D849C902C2 on vertex vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71]
2016-05-18 02:07:29,519 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:29,520 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {E85B1C0B0DF842188C1C53D849C902C2=forAllWorkUnits=true, update=[1]}
2016-05-18 02:07:29,520 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71] parallelism set to 1
2016-05-18 02:07:29,520 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 02:07:29,521 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71], numAdditionalOutputs=1
2016-05-18 02:07:29,521 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=A253BE48F71E41A7BD37C14878861CC5, vertex=vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 02:07:29,523 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for A0E40D5704B34F4BAD9BC9033C3A7F71:A253BE48F71E41A7BD37C14878861CC5 using oldmapred API
2016-05-18 02:07:29,523 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=A253BE48F71E41A7BD37C14878861CC5, vertexName=A0E40D5704B34F4BAD9BC9033C3A7F71, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 02:07:29,523 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537249337_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A0E40D5704B34F4BAD9BC9033C3A7F71, vertexId=vertex_1463537249337_0001_1_00, initRequestedTime=1463537249510, initedTime=1463537249521, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 02:07:29,524 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71] with distanceFromRoot: 0
2016-05-18 02:07:29,524 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 02:07:29,524 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537249337_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463537249337_0001_1_00, startRequestedTime=1463537249513, startedTime=1463537249524
2016-05-18 02:07:29,524 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537249337_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463537249337_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 02:07:29,524 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71] transitioned from INITED to RUNNING due to event V_START
2016-05-18 02:07:29,524 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in A0E40D5704B34F4BAD9BC9033C3A7F71
2016-05-18 02:07:29,525 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71]
2016-05-18 02:07:29,525 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537249337_0001_1][Event:TASK_STARTED]: vertexName=A0E40D5704B34F4BAD9BC9033C3A7F71, taskId=task_1463537249337_0001_1_00_000000, scheduledTime=1463537249525, launchTime=1463537249525
2016-05-18 02:07:29,525 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71]
2016-05-18 02:07:29,526 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 02:07:29,528 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463537249337_0001_00_000001, launchTime=1463537249527
2016-05-18 02:07:29,528 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537249337_0001_00_000001
2016-05-18 02:07:29,529 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463537249337_0001_1_00_000000_0] started. Is using containerId: [container_1463537249337_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 02:07:29,529 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537249337_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A0E40D5704B34F4BAD9BC9033C3A7F71, taskAttemptId=attempt_1463537249337_0001_1_00_000000_0, startTime=1463537249529, containerId=container_1463537249337_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 02:07:29,530 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537249337_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463537249337_0001_1_00_000000_0
2016-05-18 02:07:29 Starting to run new task attempt: attempt_1463537249337_0001_1_00_000000_0
2016-05-18 02:07:29,530 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463537249337_0001_1_00_000000_0
2016-05-18 02:07:29,531 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:29,532 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 02:07:29,532 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8199665CE6554C9FA27982B7D247FF9F/F61222D7BBFC4A92810F50B0E11EAE64] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1, VertexName: A0E40D5704B34F4BAD9BC9033C3A7F71, VertexParallelism: 1, TaskAttemptID:attempt_1463537249337_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E85B1C0B0DF842188C1C53D849C902C2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A253BE48F71E41A7BD37C14878861CC5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 02:07:29,532 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 02:07:29,533 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463537249337_0001_1_00_000000_0
2016-05-18 02:07:29,534 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 02:07:29,534 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 02:07:29,534 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463537249337_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 02:07:29,535 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463537249337_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 02:07:29,536 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463537249337_0001, with dag id: dag_1463537249337_0001_1
2016-05-18 02:07:29,540 INFO input.MRInput (MRInput.java:initialize(440)) - E85B1C0B0DF842188C1C53D849C902C2 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 02:07:29,540 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: E85B1C0B0DF842188C1C53D849C902C2
2016-05-18 02:07:29,541 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 02:07:29,548 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A0E40D5704B34F4BAD9BC9033C3A7F71, ordinal: 0
2016-05-18 02:07:29,548 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A0E40D5704B34F4BAD9BC9033C3A7F71, mem on start (mb), free: 126, total: 388, max: 672
2016-05-18 02:07:29,548 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 02:07:29,548 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 02:07:29,548 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 02:07:29,555 INFO output.MROutput (MROutput.java:initialize(421)) - A253BE48F71E41A7BD37C14878861CC5: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 02:07:29,556 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 02:07:29,556 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[E85B1C0B0DF842188C1C53D849C902C2:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [A253BE48F71E41A7BD37C14878861CC5:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 02:07:29,557 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 02:07:29,557 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 02:07:29,558 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 02:07:29,558 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[E85B1C0B0DF842188C1C53D849C902C2:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [A253BE48F71E41A7BD37C14878861CC5:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 02:07:29,559 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 02:07:29,559 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: E85B1C0B0DF842188C1C53D849C902C2
2016-05-18 02:07:29,559 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: E85B1C0B0DF842188C1C53D849C902C2 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 02:07:29,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 02:07:29,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 02:07:29,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 02:07:29,561 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463537249337_0001_1_00_000000_0
2016-05-18 02:07:29,562 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:0+25, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt:25+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=51}
2016-05-18 02:07:29,562 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - E85B1C0B0DF842188C1C53D849C902C2 initialized RecordReader from event
2016-05-18 02:07:29,564 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:29,565 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"] streamed: true, id: E85B1C0B0DF842188C1C53D849C902C2
2016-05-18 02:07:29,565 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_UNKNOWN_-1"], id: A253BE48F71E41A7BD37C14878861CC5
2016-05-18 02:07:29,565 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_UNKNOWN_-1"]
2016-05-18 02:07:29,566 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/nums.20.txt"]
2016-05-18 02:07:29,566 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A0E40D5704B34F4BAD9BC9033C3A7F71, all 1 inputs ready in: 00:00:00.000
2016-05-18 02:07:29,569 INFO output.MROutput (MROutput.java:close(526)) - A253BE48F71E41A7BD37C14878861CC5 closed
2016-05-18 02:07:29,569 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463537249337_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 02:07:29,570 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463537249337_0001_1_00_000000_0
2016-05-18 02:07:29,570 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A0E40D5704B34F4BAD9BC9033C3A7F71, mem on close (mb), free: 124, total: 388, max: 672
2016-05-18 02:07:29,571 INFO output.MROutput (MROutput.java:close(526)) - A253BE48F71E41A7BD37C14878861CC5 closed
2016-05-18 02:07:29,571 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 02:07:29 Completed running task attempt: attempt_1463537249337_0001_1_00_000000_0
2016-05-18 02:07:29,572 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463537249337_0001_1_00_000000_0, askedToStop=false
2016-05-18 02:07:29,572 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463537249337_0001_1_00_000000_0, stopRequested=false
2016-05-18 02:07:29,573 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463537249337_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463537249566, Process_Duration=4, Process_End_Time=1463537249570, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 02:07:29,574 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463537249337_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:29,574 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 02:07:29,574 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 02:07:29,574 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:29,574 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463537249337_0001_00_000001
2016-05-18 02:07:29,574 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537249337_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A0E40D5704B34F4BAD9BC9033C3A7F71, taskAttemptId=attempt_1463537249337_0001_1_00_000000_0, creationTime=1463537249525, allocationTime=1463537249527, startTime=1463537249529, finishTime=1463537249574, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 02:07:29,575 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463537249337_0001_1_00_000000_0
2016-05-18 02:07:29,577 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537249337_0001_1][Event:TASK_FINISHED]: vertexName=A0E40D5704B34F4BAD9BC9033C3A7F71, taskId=task_1463537249337_0001_1_00_000000, startTime=1463537249529, finishTime=1463537249575, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1463537249337_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537249566, Process_Duration=4, Process_End_Time=1463537249570, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:29,577 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 02:07:29,577 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71]
2016-05-18 02:07:29,578 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537249337_0001_1][Event:VERTEX_FINISHED]: vertexName=A0E40D5704B34F4BAD9BC9033C3A7F71, vertexId=vertex_1463537249337_0001_1_00, initRequestedTime=1463537249510, initedTime=1463537249521, startRequestedTime=1463537249513, startedTime=1463537249524, finishTime=1463537249577, timeTaken=53, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537249566, Process_Duration=4, Process_End_Time=1463537249570, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463537249529, firstTasksToStart=[ task_1463537249337_0001_1_00_000000 ], lastTaskFinishTime=1463537249574, lastTasksToFinish=[ task_1463537249337_0001_1_00_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463537249337_0001_1_00_000000 ], longestDurationTasks=[ task_1463537249337_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 02:07:29,578 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 02:07:29,578 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537249337_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463537249337_0001_00_000001, stoppedTime=1463537249578, exitStatus=0
2016-05-18 02:07:29,578 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 02:07:29,578 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:29,578 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463537249337_0001_1
2016-05-18 02:07:29,578 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537249337_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463537249337_0001_1
2016-05-18 02:07:29,579 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537249337_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 02:07:29,579 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: A253BE48F71E41A7BD37C14878861CC5 for vertex: vertex_1463537249337_0001_1_00 [A0E40D5704B34F4BAD9BC9033C3A7F71], outputName: A253BE48F71E41A7BD37C14878861CC5
2016-05-18 02:07:29,579 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463537249337_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 02:07:29,585 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463537249337_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 02:07:29,585 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463537249337_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 02:07:29,585 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463537249337_0001_00_000001, Exiting
2016-05-18 02:07:29,585 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463537249337_0001_00_000001
2016-05-18 02:07:29,585 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463537249337_0001_00_000001
2016-05-18 02:07:29,586 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463537249337_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 02:07:29,586 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463537249337_0001_00_000001 completed successfully
2016-05-18 02:07:29,587 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:A253BE48F71E41A7BD37C14878861CC5 of vertex/vertexGroup:A0E40D5704B34F4BAD9BC9033C3A7F71 isVertexGroupOutput:false
2016-05-18 02:07:29,587 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 02:07:29,589 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463537249337_0001_1, queueSize=0
2016-05-18 02:07:29,589 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463537249337_0001_1][Event:DAG_FINISHED]: dagId=dag_1463537249337_0001_1, startTime=1463537249510, finishTime=1463537249588, timeTaken=78, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=140, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463537249566, Process_Duration=4, Process_End_Time=1463537249570, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 02:07:29,589 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463537249337_0001_1 finished with state: SUCCEEDED
2016-05-18 02:07:29,589 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463537249337_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 02:07:29 Completed Dag: dag_1463537249337_0001_1
2016-05-18 02:07:29,590 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463537249337_0001_1, dagState=SUCCEEDED
2016-05-18 02:07:29,590 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463537249337_0001_1_post
2016-05-18 02:07:29,590 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 02:07:29,590 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 02:07:29,590 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 02:07:29,590 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[8199665CE6554C9FA27982B7D247FF9F/F61222D7BBFC4A92810F50B0E11EAE64] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1, with id=dag_1463537249337_0001_1
2016-05-18 02:07:29,590 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[8199665CE6554C9FA27982B7D247FF9F/F61222D7BBFC4A92810F50B0E11EAE64] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1, with id=dag_1463537249337_0001_1
2016-05-18 02:07:29,598 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1, applicationId=application_1463537249337_0001
2016-05-18 02:07:29,600 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 02:07:29,600 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 02:07:29,600 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 02:07:29,600 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 02:07:29,600 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 02:07:29,601 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 02:07:29,601 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 02:07:29,601 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 02:07:29,601 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 02:07:29,601 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 02:07:29,601 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 02:07:29,601 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 02:07:29,602 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 02:07:29,602 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 02:07:29,605 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8199665CE6554C9FA27982B7D247FF9F/F61222D7BBFC4A92810F50B0E11EAE64/.tez/application_1463537249337_0001
2016-05-18 02:07:29,606 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_UNKNOWN_-1/_temporary
2016-05-18 02:07:29,606 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.290
2016-05-18 02:07:29,697 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_UNKNOWN_-1
2016-05-18 02:07:29,721 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 02:07:29,721 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 02:07:29,750 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-18 02:07:29,753 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.006
2016-05-18 02:07:29,753 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 02:07:29,753 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 02:07:29,754 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_UNKNOWN_'field2'