2015-02-10 20:56:57,455 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(187)) - found 1 test platforms from classpath
2015-02-10 20:56:57,522 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(237)) - installing platform: hadoop2-mr1
2015-02-10 20:56:57,522 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(238)) - running test: cascading.pipe.assembly.AssemblyHelpersPlatformTest
2015-02-10 20:56:57,573 INFO hadoop2.Hadoop2MR1Platform (Hadoop2MR1Platform.java:setUp(105)) - not using cluster
2015-02-10 20:56:58,247 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-02-10 20:56:58,831 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:56:58,834 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:56:58,846 INFO property.AppProps (AppProps.java:getAppID(169)) - using app.id: 0B1F1D200D41425E936299F908462FFC
2015-02-10 20:56:58,944 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:56:58,985 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:56:58,988 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:56:59,138 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 2.6.3
2015-02-10 20:56:59,141 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] starting
2015-02-10 20:56:59,142 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:56:59,143 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:56:59,143 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false"]
2015-02-10 20:56:59,145 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] parallel execution is enabled: false
2015-02-10 20:56:59,146 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] starting jobs: 1
2015-02-10 20:56:59,146 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] allocating threads: 1
2015-02-10 20:56:59,148 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] starting step: (1/1) ...platform/multimerge+false
2015-02-10 20:56:59,172 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - session.id is deprecated. Instead, use dfs.metrics.session-id
2015-02-10 20:56:59,173 INFO jvm.JvmMetrics (JvmMetrics.java:init(76)) - Initializing JVM Metrics with processName=JobTracker, sessionId=
2015-02-10 20:56:59,198 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:56:59,502 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:56:59,513 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:56:59,544 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 20:56:59,708 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1611015894_0001
2015-02-10 20:56:59,753 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1611015894/.staging/job_local1611015894_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:56:59,755 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1611015894/.staging/job_local1611015894_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:56:59,942 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1611015894_0001/job_local1611015894_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:56:59,943 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1611015894_0001/job_local1611015894_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:56:59,955 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:56:59,959 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:56:59,959 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] submitted hadoop job: job_local1611015894_0001
2015-02-10 20:56:59,959 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] tracking url: http://localhost:8080/
2015-02-10 20:56:59,962 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:00,032 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:00,035 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1611015894_0001_m_000000_0
2015-02-10 20:57:00,076 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:00,086 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:00,086 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@55291cd
2015-02-10 20:57:00,095 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:00,102 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:00,254 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:00,255 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:00,255 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:00,255 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:00,255 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:00,280 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:00,281 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:00,342 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
2015-02-10 20:57:00,349 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:00,349 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2015-02-10 20:57:00,353 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:00,368 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:00,368 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 89, total: 202, max: 672
2015-02-10 20:57:00,368 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 30% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:00,370 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:00,370 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 89, total: 202, max: 672
2015-02-10 20:57:00,370 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 30% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:00,371 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:00,372 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 89, total: 202, max: 672
2015-02-10 20:57:00,372 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 30% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:00,373 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:00,373 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 89, total: 202, max: 672
2015-02-10 20:57:00,373 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 30% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:00,378 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:00,378 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:00,379 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:00,379 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2015-02-10 20:57:00,379 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:00,391 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:00,395 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1611015894_0001_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:00,406 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:00,406 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1611015894_0001_m_000000_0' done.
2015-02-10 20:57:00,407 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1611015894_0001_m_000000_0
2015-02-10 20:57:00,407 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1611015894_0001_m_000001_0
2015-02-10 20:57:00,409 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:00,411 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt
2015-02-10 20:57:00,412 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@47098a
2015-02-10 20:57:00,414 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:00,414 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:00,563 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:00,564 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:00,564 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:00,564 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:00,564 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:00,568 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:00,568 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:00,594 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:57:00,594 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2015-02-10 20:57:00,595 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:00,691 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:00,691 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 85, total: 302, max: 672
2015-02-10 20:57:00,691 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 44% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:00,693 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:00,693 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 85, total: 302, max: 672
2015-02-10 20:57:00,693 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 44% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:00,694 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:00,695 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 85, total: 302, max: 672
2015-02-10 20:57:00,695 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 44% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:00,696 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:00,696 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 85, total: 302, max: 672
2015-02-10 20:57:00,696 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 44% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:00,698 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:00,698 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:00,698 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:00,698 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2015-02-10 20:57:00,699 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:00,728 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:00,732 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1611015894_0001_m_000001_0 is done. And is in the process of committing
2015-02-10 20:57:00,734 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 20:57:00,735 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1611015894_0001_m_000001_0' done.
2015-02-10 20:57:00,735 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1611015894_0001_m_000001_0
2015-02-10 20:57:00,735 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:00,738 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:00,738 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1611015894_0001_r_000000_0
2015-02-10 20:57:00,747 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:00,750 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@3739491b
2015-02-10 20:57:00,764 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:00,768 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1611015894_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:00,799 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#1 about to shuffle output of map attempt_local1611015894_0001_m_000000_0 decomp: 236 len: 240 to MEMORY
2015-02-10 20:57:00,804 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local1611015894_0001_m_000000_0
2015-02-10 20:57:00,845 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->236
2015-02-10 20:57:00,849 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#1 about to shuffle output of map attempt_local1611015894_0001_m_000001_0 decomp: 236 len: 240 to MEMORY
2015-02-10 20:57:00,850 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local1611015894_0001_m_000001_0
2015-02-10 20:57:00,850 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 2, commitMemory -> 236, usedMemory ->472
2015-02-10 20:57:00,851 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:00,852 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:00,853 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:00,863 INFO mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2015-02-10 20:57:00,863 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 454 bytes
2015-02-10 20:57:00,867 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 472 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:00,867 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 474 bytes from disk
2015-02-10 20:57:00,868 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:00,868 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:00,869 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 461 bytes
2015-02-10 20:57:00,870 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:00,872 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:00,873 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:00,900 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2015-02-10 20:57:00,900 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false"]
2015-02-10 20:57:00,929 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1611015894_0001_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:00,931 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:00,931 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1611015894_0001_r_000000_0 is allowed to commit now
2015-02-10 20:57:00,933 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1611015894_0001_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false/_temporary/0/task_local1611015894_0001_r_000000
2015-02-10 20:57:00,934 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:00,934 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1611015894_0001_r_000000_0' done.
2015-02-10 20:57:00,935 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1611015894_0001_r_000000_0
2015-02-10 20:57:00,935 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:00,987 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false/_temporary
2015-02-10 20:57:01,042 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:01,083 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:01,087 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/multimerge+false
2015-02-10 20:57:01,124 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:01,124 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:01,141 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:01,147 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:01,148 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:01,164 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting
2015-02-10 20:57:01,165 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:01,166 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] sink: Hfs["TextDelimited[['num', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring"]
2015-02-10 20:57:01,167 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] parallel execution is enabled: false
2015-02-10 20:57:01,167 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting jobs: 1
2015-02-10 20:57:01,168 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] allocating threads: 1
2015-02-10 20:57:01,169 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...lpersplatform/minbystring
2015-02-10 20:57:01,171 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:01,173 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:01,208 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:01,234 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:01,271 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local532599968_0002
2015-02-10 20:57:01,293 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity532599968/.staging/job_local532599968_0002/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:01,294 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity532599968/.staging/job_local532599968_0002/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:01,386 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local532599968_0002/job_local532599968_0002.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:01,386 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local532599968_0002/job_local532599968_0002.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:01,388 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:01,388 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local532599968_0002
2015-02-10 20:57:01,389 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:01,389 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2015-02-10 20:57:01,389 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:01,394 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:01,394 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local532599968_0002_m_000000_0
2015-02-10 20:57:01,397 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:01,400 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:01,400 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4c6030f9
2015-02-10 20:57:01,405 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:01,405 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:01,557 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:01,558 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:01,558 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:01,558 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:01,558 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:01,561 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:01,561 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:01,575 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:01,575 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'num']]
2015-02-10 20:57:01,576 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:01,578 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:01,578 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 89, total: 402, max: 672
2015-02-10 20:57:01,578 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 59% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:01,579 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:01,579 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 89, total: 402, max: 672
2015-02-10 20:57:01,580 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 59% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:01,582 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:01,582 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:01,582 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:01,582 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-10 20:57:01,582 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:01,587 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:01,590 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local532599968_0002_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:01,593 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:01,593 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local532599968_0002_m_000000_0' done.
2015-02-10 20:57:01,594 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local532599968_0002_m_000000_0
2015-02-10 20:57:01,594 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:01,595 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:01,595 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local532599968_0002_r_000000_0
2015-02-10 20:57:01,599 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:01,599 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5637dde9
2015-02-10 20:57:01,600 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:01,602 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local532599968_0002_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:01,605 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#2 about to shuffle output of map attempt_local532599968_0002_m_000000_0 decomp: 87 len: 91 to MEMORY
2015-02-10 20:57:01,606 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local532599968_0002_m_000000_0
2015-02-10 20:57:01,606 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2015-02-10 20:57:01,607 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:01,609 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:01,609 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:01,613 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:01,614 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 20:57:01,616 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 87 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:01,617 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 91 bytes from disk
2015-02-10 20:57:01,618 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:01,618 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:01,619 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 20:57:01,620 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:01,622 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:01,622 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:01,635 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'num']]
2015-02-10 20:57:01,635 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring"]
2015-02-10 20:57:01,652 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local532599968_0002_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:01,654 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:01,654 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local532599968_0002_r_000000_0 is allowed to commit now
2015-02-10 20:57:01,656 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local532599968_0002_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring/_temporary/0/task_local532599968_0002_r_000000
2015-02-10 20:57:01,658 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:01,658 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local532599968_0002_r_000000_0' done.
2015-02-10 20:57:01,659 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local532599968_0002_r_000000_0
2015-02-10 20:57:01,660 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:01,688 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring/_temporary
2015-02-10 20:57:01,730 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:01,775 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:01,778 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/minbystring
2015-02-10 20:57:01,818 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:01,819 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:01,853 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:01,862 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:01,863 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:01,879 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs] starting
2015-02-10 20:57:01,879 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:57:01,879 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:01,880 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs] sink: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage"]
2015-02-10 20:57:01,880 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs] parallel execution is enabled: false
2015-02-10 20:57:01,880 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs] starting jobs: 1
2015-02-10 20:57:01,880 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs] allocating threads: 1
2015-02-10 20:57:01,881 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average-lhs+average-rhs] starting step: (1/1) ...persplatform/mergeaverage
2015-02-10 20:57:01,882 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:01,884 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:01,918 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:01,920 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:01,944 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 20:57:01,984 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local115791930_0003
2015-02-10 20:57:02,012 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity115791930/.staging/job_local115791930_0003/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:02,012 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity115791930/.staging/job_local115791930_0003/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:02,086 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local115791930_0003/job_local115791930_0003.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:02,086 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local115791930_0003/job_local115791930_0003.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:02,088 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:02,088 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average-lhs+average-rhs] submitted hadoop job: job_local115791930_0003
2015-02-10 20:57:02,088 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average-lhs+average-rhs] tracking url: http://localhost:8080/
2015-02-10 20:57:02,090 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:02,090 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:02,096 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:02,096 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local115791930_0003_m_000000_0
2015-02-10 20:57:02,100 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:02,103 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:02,103 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@12f53870
2015-02-10 20:57:02,106 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:02,107 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:02,263 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:02,264 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:02,264 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:02,265 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:02,265 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:02,269 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:02,270 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:02,286 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:02,287 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2015-02-10 20:57:02,288 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:02,290 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:02,291 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 73, total: 502, max: 672
2015-02-10 20:57:02,291 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 74% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:02,293 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:02,293 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 73, total: 502, max: 672
2015-02-10 20:57:02,294 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 74% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:02,295 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:02,295 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 73, total: 502, max: 672
2015-02-10 20:57:02,296 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 74% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:02,297 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:02,297 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 73, total: 502, max: 672
2015-02-10 20:57:02,298 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 74% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:02,300 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:02,301 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:02,301 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:02,302 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2015-02-10 20:57:02,302 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:02,308 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:02,325 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local115791930_0003_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:02,329 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:02,330 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local115791930_0003_m_000000_0' done.
2015-02-10 20:57:02,330 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local115791930_0003_m_000000_0
2015-02-10 20:57:02,331 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local115791930_0003_m_000001_0
2015-02-10 20:57:02,333 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:02,336 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt
2015-02-10 20:57:02,336 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6f539d49
2015-02-10 20:57:02,338 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:02,339 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:02,483 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:02,484 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:02,485 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:02,485 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:02,486 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:02,487 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:02,488 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:02,504 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:57:02,505 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2015-02-10 20:57:02,506 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:02,511 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:02,511 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 54, total: 560, max: 672
2015-02-10 20:57:02,513 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:02,513 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 54, total: 560, max: 672
2015-02-10 20:57:02,515 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:02,515 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 54, total: 560, max: 672
2015-02-10 20:57:02,516 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:02,517 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 54, total: 560, max: 672
2015-02-10 20:57:02,519 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:02,519 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:02,520 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:02,520 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2015-02-10 20:57:02,521 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:02,526 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:02,528 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local115791930_0003_m_000001_0 is done. And is in the process of committing
2015-02-10 20:57:02,531 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 20:57:02,532 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local115791930_0003_m_000001_0' done.
2015-02-10 20:57:02,533 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local115791930_0003_m_000001_0
2015-02-10 20:57:02,533 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:02,534 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:02,534 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local115791930_0003_r_000000_0
2015-02-10 20:57:02,537 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:02,538 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@3a5e7663
2015-02-10 20:57:02,539 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:02,541 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local115791930_0003_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:02,543 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#3 about to shuffle output of map attempt_local115791930_0003_m_000001_0 decomp: 200 len: 204 to MEMORY
2015-02-10 20:57:02,544 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local115791930_0003_m_000001_0
2015-02-10 20:57:02,544 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->200
2015-02-10 20:57:02,547 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#3 about to shuffle output of map attempt_local115791930_0003_m_000000_0 decomp: 200 len: 204 to MEMORY
2015-02-10 20:57:02,548 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local115791930_0003_m_000000_0
2015-02-10 20:57:02,548 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 2, commitMemory -> 200, usedMemory ->400
2015-02-10 20:57:02,549 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:02,550 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:02,551 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:02,554 INFO mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2015-02-10 20:57:02,555 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 382 bytes
2015-02-10 20:57:02,558 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 400 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:02,558 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 402 bytes from disk
2015-02-10 20:57:02,559 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:02,560 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:02,560 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 389 bytes
2015-02-10 20:57:02,561 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:02,562 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:02,563 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:02,577 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2015-02-10 20:57:02,578 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage"]
2015-02-10 20:57:02,593 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local115791930_0003_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:02,595 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:02,595 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local115791930_0003_r_000000_0 is allowed to commit now
2015-02-10 20:57:02,597 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local115791930_0003_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage/_temporary/0/task_local115791930_0003_r_000000
2015-02-10 20:57:02,599 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:02,599 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local115791930_0003_r_000000_0' done.
2015-02-10 20:57:02,600 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local115791930_0003_r_000000_0
2015-02-10 20:57:02,601 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:02,633 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage/_temporary
2015-02-10 20:57:02,671 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:02,711 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:02,713 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/mergeaverage
2015-02-10 20:57:02,744 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:02,745 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:02,757 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:02,761 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:02,762 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:02,776 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 20:57:02,777 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:02,777 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] sink: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall"]
2015-02-10 20:57:02,778 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] parallel execution is enabled: false
2015-02-10 20:57:02,778 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting jobs: 1
2015-02-10 20:57:02,779 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] allocating threads: 1
2015-02-10 20:57:02,780 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...helpersplatform/renameall
2015-02-10 20:57:02,781 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:02,783 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:02,819 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:02,846 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:02,870 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local742608088_0004
2015-02-10 20:57:02,890 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity742608088/.staging/job_local742608088_0004/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:02,891 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity742608088/.staging/job_local742608088_0004/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:02,970 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local742608088_0004/job_local742608088_0004.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:02,971 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local742608088_0004/job_local742608088_0004.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:02,973 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:02,974 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local742608088_0004
2015-02-10 20:57:02,974 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:02,975 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:02,975 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 20:57:02,980 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:02,981 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local742608088_0004_m_000000_0
2015-02-10 20:57:02,984 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:02,986 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt
2015-02-10 20:57:02,986 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@b8235a1
2015-02-10 20:57:02,988 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 20:57:02,999 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:02,999 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:03,007 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:03,008 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall"]
2015-02-10 20:57:03,012 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:03,012 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local742608088_0004_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:03,014 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:03,014 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local742608088_0004_m_000000_0 is allowed to commit now
2015-02-10 20:57:03,015 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local742608088_0004_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall/_temporary/0/task_local742608088_0004_m_000000
2015-02-10 20:57:03,017 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt:0+19
2015-02-10 20:57:03,017 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local742608088_0004_m_000000_0' done.
2015-02-10 20:57:03,017 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local742608088_0004_m_000000_0
2015-02-10 20:57:03,019 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:03,047 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall/_temporary
2015-02-10 20:57:03,396 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:03,399 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/renameall
2015-02-10 20:57:03,427 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:03,427 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:03,448 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:03,452 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:03,468 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:03,478 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting
2015-02-10 20:57:03,479 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:03,480 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] sink: Hfs["TextDelimited[['count', 'count2']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount"]
2015-02-10 20:57:03,480 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] parallel execution is enabled: false
2015-02-10 20:57:03,480 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting jobs: 2
2015-02-10 20:57:03,480 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] allocating threads: 1
2015-02-10 20:57:03,482 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/2)
2015-02-10 20:57:03,483 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:03,484 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:03,513 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:03,535 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:03,554 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local613160317_0005
2015-02-10 20:57:03,573 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity613160317/.staging/job_local613160317_0005/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:03,574 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity613160317/.staging/job_local613160317_0005/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:03,629 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local613160317_0005/job_local613160317_0005.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:03,629 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local613160317_0005/job_local613160317_0005.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:03,630 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:03,631 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local613160317_0005
2015-02-10 20:57:03,631 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-10 20:57:03,631 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:03,631 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:03,635 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:03,635 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local613160317_0005_m_000000_0
2015-02-10 20:57:03,637 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:03,639 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:03,639 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1259b924
2015-02-10 20:57:03,641 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:03,641 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:03,712 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:03,712 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:03,712 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:03,712 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:03,713 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:03,715 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:03,716 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:03,728 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:03,728 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{1}:'char']]
2015-02-10 20:57:03,728 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:03,730 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:03,730 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 81, total: 220, max: 672
2015-02-10 20:57:03,730 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:03,731 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:03,732 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 81, total: 220, max: 672
2015-02-10 20:57:03,732 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:03,732 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:03,732 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 81, total: 220, max: 672
2015-02-10 20:57:03,733 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:03,733 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:03,733 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 81, total: 220, max: 672
2015-02-10 20:57:03,733 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:03,735 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:03,735 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:03,735 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:03,736 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 20:57:03,736 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:03,740 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:03,742 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local613160317_0005_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:03,745 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:03,745 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local613160317_0005_m_000000_0' done.
2015-02-10 20:57:03,746 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local613160317_0005_m_000000_0
2015-02-10 20:57:03,747 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:03,748 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:03,749 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local613160317_0005_r_000000_0
2015-02-10 20:57:03,754 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:03,754 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7ab05cd7
2015-02-10 20:57:03,755 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:03,755 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local613160317_0005_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:03,757 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#4 about to shuffle output of map attempt_local613160317_0005_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 20:57:03,758 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local613160317_0005_m_000000_0
2015-02-10 20:57:03,758 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 20:57:03,759 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:03,760 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:03,760 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:03,763 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:03,764 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:03,766 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:03,766 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 20:57:03,766 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:03,766 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:03,767 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:03,768 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:03,768 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:03,768 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:03,779 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{1}:'char']]
2015-02-10 20:57:03,779 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['count', 'count2']]"][4331726180/count/]
2015-02-10 20:57:03,802 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:03,805 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:03,806 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 77, total: 220, max: 672
2015-02-10 20:57:03,806 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:03,806 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local613160317_0005_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:03,809 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:03,809 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local613160317_0005_r_000000_0 is allowed to commit now
2015-02-10 20:57:03,810 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local613160317_0005_r_000000_0' to file:/tmp/hadoop-teamcity/4331726180_count_DFACB92F77D44369982ED63DE0178E64/_temporary/0/task_local613160317_0005_r_000000
2015-02-10 20:57:03,812 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:03,812 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local613160317_0005_r_000000_0' done.
2015-02-10 20:57:03,812 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local613160317_0005_r_000000_0
2015-02-10 20:57:03,812 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:03,833 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (2/2) ...elpersplatform/countcount
2015-02-10 20:57:03,833 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:03,835 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:03,875 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:03,895 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:03,907 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:03,907 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:03,919 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1429604986_0006
2015-02-10 20:57:03,940 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1429604986/.staging/job_local1429604986_0006/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:03,940 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1429604986/.staging/job_local1429604986_0006/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:03,996 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1429604986_0006/job_local1429604986_0006.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:03,996 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1429604986_0006/job_local1429604986_0006.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:03,997 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:03,998 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local1429604986_0006
2015-02-10 20:57:03,998 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:03,998 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-10 20:57:03,999 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:04,003 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:04,004 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1429604986_0006_m_000000_0
2015-02-10 20:57:04,006 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:04,008 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/4331726180_count_DFACB92F77D44369982ED63DE0178E64/part-00000
2015-02-10 20:57:04,008 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@60f00e0f
2015-02-10 20:57:04,019 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:04,020 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:04,186 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:04,187 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:04,187 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:04,187 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:04,187 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:04,188 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:04,189 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:04,200 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['count', 'count2']]"][4331726180/count/]
2015-02-10 20:57:04,200 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{1}:'count']]
2015-02-10 20:57:04,203 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:04,203 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:04,204 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:04,204 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 21; bufvoid = 104857600
2015-02-10 20:57:04,204 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214388(104857552); length = 9/6553600
2015-02-10 20:57:04,208 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:04,210 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1429604986_0006_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:04,212 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/4331726180_count_DFACB92F77D44369982ED63DE0178E64/part-00000:0+112
2015-02-10 20:57:04,213 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1429604986_0006_m_000000_0' done.
2015-02-10 20:57:04,213 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1429604986_0006_m_000000_0
2015-02-10 20:57:04,213 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:04,214 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:04,214 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1429604986_0006_r_000000_0
2015-02-10 20:57:04,216 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:04,216 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@6e39b27b
2015-02-10 20:57:04,218 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:04,219 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1429604986_0006_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:04,221 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#5 about to shuffle output of map attempt_local1429604986_0006_m_000000_0 decomp: 29 len: 33 to MEMORY
2015-02-10 20:57:04,222 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 29 bytes from map-output for attempt_local1429604986_0006_m_000000_0
2015-02-10 20:57:04,222 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 29, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->29
2015-02-10 20:57:04,222 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:04,223 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:04,224 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:04,230 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:04,230 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 24 bytes
2015-02-10 20:57:04,232 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 29 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:04,233 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 33 bytes from disk
2015-02-10 20:57:04,233 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:04,233 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:04,234 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 24 bytes
2015-02-10 20:57:04,235 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:04,236 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:04,236 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:04,248 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{1}:'count']]
2015-02-10 20:57:04,248 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['count', 'count2']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount"]
2015-02-10 20:57:04,259 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1429604986_0006_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:04,261 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:04,261 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1429604986_0006_r_000000_0 is allowed to commit now
2015-02-10 20:57:04,262 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1429604986_0006_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount/_temporary/0/task_local1429604986_0006_r_000000
2015-02-10 20:57:04,266 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:04,266 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1429604986_0006_r_000000_0' done.
2015-02-10 20:57:04,266 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1429604986_0006_r_000000_0
2015-02-10 20:57:04,266 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:04,292 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount/_temporary
2015-02-10 20:57:04,326 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:04,359 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:04,361 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/countcount
2015-02-10 20:57:04,391 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:04,391 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:04,408 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:04,412 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:04,413 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:04,425 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs] starting
2015-02-10 20:57:04,426 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:57:04,426 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:04,426 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs] sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount"]
2015-02-10 20:57:04,427 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs] parallel execution is enabled: false
2015-02-10 20:57:04,427 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs] starting jobs: 1
2015-02-10 20:57:04,427 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs] allocating threads: 1
2015-02-10 20:57:04,427 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs+count-rhs] starting step: (1/1) ...elpersplatform/mergecount
2015-02-10 20:57:04,428 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:04,430 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:04,458 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:04,460 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:04,480 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 20:57:04,499 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local606478226_0007
2015-02-10 20:57:04,522 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity606478226/.staging/job_local606478226_0007/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:04,523 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity606478226/.staging/job_local606478226_0007/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:04,583 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local606478226_0007/job_local606478226_0007.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:04,583 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local606478226_0007/job_local606478226_0007.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:04,584 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:04,584 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs+count-rhs] submitted hadoop job: job_local606478226_0007
2015-02-10 20:57:04,585 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:04,585 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs+count-rhs] tracking url: http://localhost:8080/
2015-02-10 20:57:04,585 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:04,588 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:04,588 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local606478226_0007_m_000000_0
2015-02-10 20:57:04,590 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:04,592 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:04,592 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6d9dd520
2015-02-10 20:57:04,594 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:04,594 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:04,740 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:04,740 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:04,740 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:04,741 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:04,741 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:04,743 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:04,743 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:04,757 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:04,757 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 20:57:04,758 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:04,760 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:04,760 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 114, total: 431, max: 672
2015-02-10 20:57:04,760 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 64% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:04,761 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:04,761 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 114, total: 431, max: 672
2015-02-10 20:57:04,761 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 64% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:04,762 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:04,762 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 114, total: 431, max: 672
2015-02-10 20:57:04,762 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 64% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:04,763 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:04,763 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 114, total: 431, max: 672
2015-02-10 20:57:04,763 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 64% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:04,765 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:04,765 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:04,765 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:04,765 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 20:57:04,766 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:04,770 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:04,772 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local606478226_0007_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:04,775 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:04,775 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local606478226_0007_m_000000_0' done.
2015-02-10 20:57:04,775 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local606478226_0007_m_000000_0
2015-02-10 20:57:04,775 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local606478226_0007_m_000001_0
2015-02-10 20:57:04,780 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:04,782 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt
2015-02-10 20:57:04,782 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@642b6fc7
2015-02-10 20:57:04,784 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:04,784 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:04,928 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:04,928 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:04,929 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:04,929 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:04,929 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:04,931 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:04,931 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:04,946 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:57:04,946 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 20:57:04,947 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:04,951 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:04,951 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 113, total: 531, max: 672
2015-02-10 20:57:04,952 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:04,953 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:04,953 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 113, total: 531, max: 672
2015-02-10 20:57:04,953 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:04,954 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:04,954 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 113, total: 531, max: 672
2015-02-10 20:57:04,954 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:04,955 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:04,955 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 113, total: 531, max: 672
2015-02-10 20:57:04,955 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:04,956 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:04,957 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:04,957 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:04,957 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 20:57:04,957 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:04,961 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:04,964 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local606478226_0007_m_000001_0 is done. And is in the process of committing
2015-02-10 20:57:04,966 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 20:57:04,966 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local606478226_0007_m_000001_0' done.
2015-02-10 20:57:04,966 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local606478226_0007_m_000001_0
2015-02-10 20:57:04,967 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:04,967 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:04,967 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local606478226_0007_r_000000_0
2015-02-10 20:57:04,970 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:04,971 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@4aa4ceeb
2015-02-10 20:57:04,972 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:04,973 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local606478226_0007_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:04,976 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#6 about to shuffle output of map attempt_local606478226_0007_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 20:57:04,977 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local606478226_0007_m_000000_0
2015-02-10 20:57:04,977 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 20:57:04,980 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#6 about to shuffle output of map attempt_local606478226_0007_m_000001_0 decomp: 119 len: 123 to MEMORY
2015-02-10 20:57:04,981 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local606478226_0007_m_000001_0
2015-02-10 20:57:04,981 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 2, commitMemory -> 119, usedMemory ->238
2015-02-10 20:57:04,981 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:04,982 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:04,983 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:04,985 INFO mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2015-02-10 20:57:04,985 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 220 bytes
2015-02-10 20:57:04,988 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 238 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:04,988 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 240 bytes from disk
2015-02-10 20:57:04,988 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:04,988 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:04,989 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 227 bytes
2015-02-10 20:57:04,990 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:04,990 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:04,991 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:05,004 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 20:57:05,004 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount"]
2015-02-10 20:57:05,017 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local606478226_0007_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:05,019 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:05,019 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local606478226_0007_r_000000_0 is allowed to commit now
2015-02-10 20:57:05,020 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local606478226_0007_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount/_temporary/0/task_local606478226_0007_r_000000
2015-02-10 20:57:05,022 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:05,022 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local606478226_0007_r_000000_0' done.
2015-02-10 20:57:05,022 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local606478226_0007_r_000000_0
2015-02-10 20:57:05,022 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:05,053 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount/_temporary
2015-02-10 20:57:05,086 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:05,122 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:05,124 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/mergecount
2015-02-10 20:57:05,149 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:05,150 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:05,161 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:05,164 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:05,165 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:05,175 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] starting
2015-02-10 20:57:05,175 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:05,175 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] sink: Hfs["TextDelimited[['char', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbynullsafety"]
2015-02-10 20:57:05,176 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] parallel execution is enabled: false
2015-02-10 20:57:05,176 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] starting jobs: 1
2015-02-10 20:57:05,176 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] allocating threads: 1
2015-02-10 20:57:05,177 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] starting step: (1/1) ...splatform/minbynullsafety
2015-02-10 20:57:05,177 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:05,178 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:05,207 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:05,228 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:05,246 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local882655297_0008
2015-02-10 20:57:05,265 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity882655297/.staging/job_local882655297_0008/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:05,266 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity882655297/.staging/job_local882655297_0008/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:05,331 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local882655297_0008/job_local882655297_0008.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:05,331 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local882655297_0008/job_local882655297_0008.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:05,333 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:05,334 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:05,334 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:05,334 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] submitted hadoop job: job_local882655297_0008
2015-02-10 20:57:05,335 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] tracking url: http://localhost:8080/
2015-02-10 20:57:05,338 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:05,338 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local882655297_0008_m_000000_0
2015-02-10 20:57:05,341 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:05,343 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:05,343 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@620a9239
2015-02-10 20:57:05,344 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:05,345 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:05,671 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:05,671 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:05,671 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:05,672 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:05,672 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:05,673 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:05,673 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:05,684 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:05,685 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(min)[by:[{1}:'char']]
2015-02-10 20:57:05,685 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:05,687 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:05,687 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 148, total: 254, max: 672
2015-02-10 20:57:05,687 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 37% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:05,688 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:05,689 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 148, total: 254, max: 672
2015-02-10 20:57:05,689 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 37% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:05,690 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:05,690 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 148, total: 254, max: 672
2015-02-10 20:57:05,690 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 37% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:05,690 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:05,691 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 148, total: 254, max: 672
2015-02-10 20:57:05,691 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 37% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:05,693 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:05,694 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:05,694 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:05,694 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 20:57:05,694 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:05,737 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:05,739 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local882655297_0008_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:05,742 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:05,742 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local882655297_0008_m_000000_0' done.
2015-02-10 20:57:05,742 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local882655297_0008_m_000000_0
2015-02-10 20:57:05,742 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:05,743 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:05,743 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local882655297_0008_r_000000_0
2015-02-10 20:57:05,745 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:05,745 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@bab5e85
2015-02-10 20:57:05,746 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:05,747 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local882655297_0008_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:05,749 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#7 about to shuffle output of map attempt_local882655297_0008_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 20:57:05,749 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local882655297_0008_m_000000_0
2015-02-10 20:57:05,749 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 20:57:05,750 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:05,751 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:05,752 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:05,754 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:05,754 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:05,756 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:05,756 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 20:57:05,756 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:05,757 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:05,757 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:05,758 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:05,758 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:05,758 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:05,768 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(min)[by:[{1}:'char']]
2015-02-10 20:57:05,769 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbynullsafety"]
2015-02-10 20:57:05,780 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local882655297_0008_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:05,782 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:05,782 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local882655297_0008_r_000000_0 is allowed to commit now
2015-02-10 20:57:05,783 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local882655297_0008_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbynullsafety/_temporary/0/task_local882655297_0008_r_000000
2015-02-10 20:57:05,785 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:05,785 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local882655297_0008_r_000000_0' done.
2015-02-10 20:57:05,785 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local882655297_0008_r_000000_0
2015-02-10 20:57:05,785 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:05,810 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbynullsafety/_temporary
2015-02-10 20:57:05,843 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:05,878 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:05,880 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/minbynullsafety
2015-02-10 20:57:05,905 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:05,905 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:05,915 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:05,917 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:05,918 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:05,929 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting
2015-02-10 20:57:05,929 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:05,929 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] sink: Hfs["TextDelimited[['num', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring"]
2015-02-10 20:57:05,929 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] parallel execution is enabled: false
2015-02-10 20:57:05,929 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting jobs: 1
2015-02-10 20:57:05,929 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] allocating threads: 1
2015-02-10 20:57:05,930 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...lpersplatform/maxbystring
2015-02-10 20:57:05,931 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:05,932 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:05,959 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:05,981 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:05,998 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1776200430_0009
2015-02-10 20:57:06,016 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1776200430/.staging/job_local1776200430_0009/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:06,016 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1776200430/.staging/job_local1776200430_0009/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:06,069 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1776200430_0009/job_local1776200430_0009.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:06,070 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1776200430_0009/job_local1776200430_0009.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:06,072 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:06,072 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local1776200430_0009
2015-02-10 20:57:06,072 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2015-02-10 20:57:06,072 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:06,073 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:06,076 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:06,076 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1776200430_0009_m_000000_0
2015-02-10 20:57:06,079 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:06,081 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:06,081 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@49f8a4fd
2015-02-10 20:57:06,083 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:06,083 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:06,251 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:06,252 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:06,252 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:06,252 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:06,252 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:06,253 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:06,254 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:06,263 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:06,263 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'num']]
2015-02-10 20:57:06,264 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:06,266 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:06,266 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 101, total: 354, max: 672
2015-02-10 20:57:06,266 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 52% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:06,267 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:06,267 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 101, total: 354, max: 672
2015-02-10 20:57:06,267 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 52% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:06,269 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:06,269 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:06,269 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:06,269 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-10 20:57:06,269 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:06,273 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:06,275 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1776200430_0009_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:06,277 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:06,277 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1776200430_0009_m_000000_0' done.
2015-02-10 20:57:06,278 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1776200430_0009_m_000000_0
2015-02-10 20:57:06,278 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:06,278 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:06,279 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1776200430_0009_r_000000_0
2015-02-10 20:57:06,280 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:06,281 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@2587b08d
2015-02-10 20:57:06,281 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:06,282 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1776200430_0009_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:06,284 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#8 about to shuffle output of map attempt_local1776200430_0009_m_000000_0 decomp: 87 len: 91 to MEMORY
2015-02-10 20:57:06,285 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local1776200430_0009_m_000000_0
2015-02-10 20:57:06,286 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2015-02-10 20:57:06,287 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:06,288 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:06,288 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:06,290 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:06,290 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 20:57:06,292 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 87 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:06,292 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 91 bytes from disk
2015-02-10 20:57:06,292 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:06,293 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:06,293 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 20:57:06,294 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:06,294 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:06,294 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:06,303 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'num']]
2015-02-10 20:57:06,304 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring"]
2015-02-10 20:57:06,315 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1776200430_0009_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:06,317 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:06,317 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1776200430_0009_r_000000_0 is allowed to commit now
2015-02-10 20:57:06,318 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1776200430_0009_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring/_temporary/0/task_local1776200430_0009_r_000000
2015-02-10 20:57:06,320 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:06,320 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1776200430_0009_r_000000_0' done.
2015-02-10 20:57:06,320 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1776200430_0009_r_000000_0
2015-02-10 20:57:06,321 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:06,345 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring/_temporary
2015-02-10 20:57:06,377 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:06,416 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:06,418 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/maxbystring
2015-02-10 20:57:06,453 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:06,454 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:06,468 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:06,471 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:06,472 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:06,488 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] starting
2015-02-10 20:57:06,489 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] source: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:57:06,489 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] sink: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfields"]
2015-02-10 20:57:06,489 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] parallel execution is enabled: false
2015-02-10 20:57:06,489 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] starting jobs: 1
2015-02-10 20:57:06,489 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] allocating threads: 1
2015-02-10 20:57:06,490 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] starting step: (1/1) ...persplatform/firstnfields
2015-02-10 20:57:06,491 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:06,492 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:06,521 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:06,542 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:06,563 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1121178405_0010
2015-02-10 20:57:06,585 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1121178405/.staging/job_local1121178405_0010/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:06,585 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1121178405/.staging/job_local1121178405_0010/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:06,638 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1121178405_0010/job_local1121178405_0010.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:06,639 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1121178405_0010/job_local1121178405_0010.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:06,640 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:06,640 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] submitted hadoop job: job_local1121178405_0010
2015-02-10 20:57:06,640 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:06,640 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] tracking url: http://localhost:8080/
2015-02-10 20:57:06,642 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:06,645 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:06,646 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1121178405_0010_m_000000_0
2015-02-10 20:57:06,648 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:06,650 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:57:06,650 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5054c2b8
2015-02-10 20:57:06,652 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:06,652 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:06,797 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:06,797 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:06,798 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:06,798 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:06,798 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:06,802 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:06,802 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:06,812 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:57:06,813 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(first)[by:[{1}:'num']]
2015-02-10 20:57:06,813 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:06,817 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:06,817 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 130, total: 454, max: 672
2015-02-10 20:57:06,818 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 67% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:06,819 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:06,820 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 130, total: 454, max: 672
2015-02-10 20:57:06,820 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 67% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:06,822 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:06,823 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:06,823 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:06,823 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 170; bufvoid = 104857600
2015-02-10 20:57:06,824 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:06,827 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:06,829 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1121178405_0010_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:06,832 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:57:06,832 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1121178405_0010_m_000000_0' done.
2015-02-10 20:57:06,832 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1121178405_0010_m_000000_0
2015-02-10 20:57:06,833 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:06,833 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:06,833 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1121178405_0010_r_000000_0
2015-02-10 20:57:06,835 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:06,836 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7e628e42
2015-02-10 20:57:06,836 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:06,837 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1121178405_0010_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:06,839 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#9 about to shuffle output of map attempt_local1121178405_0010_m_000000_0 decomp: 182 len: 186 to MEMORY
2015-02-10 20:57:06,840 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 182 bytes from map-output for attempt_local1121178405_0010_m_000000_0
2015-02-10 20:57:06,840 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 182, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->182
2015-02-10 20:57:06,841 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:06,842 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:06,842 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:06,845 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:06,845 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 160 bytes
2015-02-10 20:57:06,846 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 182 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:06,847 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 186 bytes from disk
2015-02-10 20:57:06,847 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:06,847 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:06,848 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 160 bytes
2015-02-10 20:57:06,848 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:06,849 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:06,849 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:06,858 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(first)[by:[{1}:'num']]
2015-02-10 20:57:06,859 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfields"]
2015-02-10 20:57:06,870 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1121178405_0010_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:06,872 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:06,872 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1121178405_0010_r_000000_0 is allowed to commit now
2015-02-10 20:57:06,873 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1121178405_0010_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfields/_temporary/0/task_local1121178405_0010_r_000000
2015-02-10 20:57:06,875 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:06,875 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1121178405_0010_r_000000_0' done.
2015-02-10 20:57:06,875 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1121178405_0010_r_000000_0
2015-02-10 20:57:06,875 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:06,895 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfields/_temporary
2015-02-10 20:57:06,926 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:06,928 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/firstnfields
2015-02-10 20:57:06,954 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:06,955 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:06,963 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:06,965 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:06,966 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:06,975 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] starting
2015-02-10 20:57:06,975 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:06,976 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] sink: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields"]
2015-02-10 20:57:06,976 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] parallel execution is enabled: false
2015-02-10 20:57:06,976 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] starting jobs: 1
2015-02-10 20:57:06,976 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] allocating threads: 1
2015-02-10 20:57:06,978 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] starting step: (1/1) ...persplatform/coercefields
2015-02-10 20:57:06,979 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:06,980 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:07,006 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:07,026 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:07,042 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1599758861_0011
2015-02-10 20:57:07,060 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1599758861/.staging/job_local1599758861_0011/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:07,061 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1599758861/.staging/job_local1599758861_0011/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:07,114 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1599758861_0011/job_local1599758861_0011.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:07,114 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1599758861_0011/job_local1599758861_0011.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:07,115 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:07,116 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] submitted hadoop job: job_local1599758861_0011
2015-02-10 20:57:07,116 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:07,116 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] tracking url: http://localhost:8080/
2015-02-10 20:57:07,117 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:07,119 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:07,120 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1599758861_0011_m_000000_0
2015-02-10 20:57:07,122 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:07,123 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt
2015-02-10 20:57:07,124 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@38128f03
2015-02-10 20:57:07,125 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 20:57:07,133 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:07,133 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:07,143 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:07,143 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields"]
2015-02-10 20:57:07,154 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:07,154 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1599758861_0011_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:07,155 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:07,156 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1599758861_0011_m_000000_0 is allowed to commit now
2015-02-10 20:57:07,156 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1599758861_0011_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields/_temporary/0/task_local1599758861_0011_m_000000
2015-02-10 20:57:07,158 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt:0+19
2015-02-10 20:57:07,158 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1599758861_0011_m_000000_0' done.
2015-02-10 20:57:07,158 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1599758861_0011_m_000000_0
2015-02-10 20:57:07,158 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:07,182 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields/_temporary
2015-02-10 20:57:07,212 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:07,214 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/coercefields
2015-02-10 20:57:07,241 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:07,242 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:07,253 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:07,255 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:07,256 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:07,263 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting
2015-02-10 20:57:07,263 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:07,263 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count"]
2015-02-10 20:57:07,264 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] parallel execution is enabled: false
2015-02-10 20:57:07,264 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting jobs: 1
2015-02-10 20:57:07,264 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] allocating threads: 1
2015-02-10 20:57:07,264 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...mblyhelpersplatform/count
2015-02-10 20:57:07,265 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:07,267 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:07,297 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:07,319 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:07,334 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local518708628_0012
2015-02-10 20:57:07,357 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity518708628/.staging/job_local518708628_0012/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:07,357 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity518708628/.staging/job_local518708628_0012/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:07,411 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local518708628_0012/job_local518708628_0012.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:07,411 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local518708628_0012/job_local518708628_0012.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:07,413 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:07,413 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local518708628_0012
2015-02-10 20:57:07,413 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:07,414 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-10 20:57:07,414 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:07,417 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:07,417 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local518708628_0012_m_000000_0
2015-02-10 20:57:07,418 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:07,420 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:07,420 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@e26fb32
2015-02-10 20:57:07,421 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:07,422 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:07,564 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:07,564 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:07,564 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:07,565 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:07,565 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:07,566 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:07,566 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:07,576 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:07,576 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{1}:'char']]
2015-02-10 20:57:07,576 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:07,578 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:07,578 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 160, total: 590, max: 672
2015-02-10 20:57:07,579 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:07,579 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 160, total: 590, max: 672
2015-02-10 20:57:07,580 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:07,580 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 160, total: 590, max: 672
2015-02-10 20:57:07,581 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:07,581 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 160, total: 590, max: 672
2015-02-10 20:57:07,582 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:07,582 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:07,582 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:07,583 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 20:57:07,583 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:07,588 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:07,590 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local518708628_0012_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:07,592 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:07,592 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local518708628_0012_m_000000_0' done.
2015-02-10 20:57:07,592 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local518708628_0012_m_000000_0
2015-02-10 20:57:07,592 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:07,593 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:07,593 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local518708628_0012_r_000000_0
2015-02-10 20:57:07,595 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:07,595 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7b14b3af
2015-02-10 20:57:07,596 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:07,597 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local518708628_0012_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:07,599 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#10 about to shuffle output of map attempt_local518708628_0012_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 20:57:07,599 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local518708628_0012_m_000000_0
2015-02-10 20:57:07,600 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 20:57:07,600 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:07,601 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:07,601 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:07,603 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:07,604 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:07,605 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:07,606 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 20:57:07,606 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:07,606 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:07,606 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:07,607 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:07,607 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:07,607 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:07,616 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{1}:'char']]
2015-02-10 20:57:07,618 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count"]
2015-02-10 20:57:07,630 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local518708628_0012_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:07,631 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:07,632 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local518708628_0012_r_000000_0 is allowed to commit now
2015-02-10 20:57:07,633 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local518708628_0012_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count/_temporary/0/task_local518708628_0012_r_000000
2015-02-10 20:57:07,634 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:07,635 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local518708628_0012_r_000000_0' done.
2015-02-10 20:57:07,635 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local518708628_0012_r_000000_0
2015-02-10 20:57:07,635 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:07,656 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count/_temporary
2015-02-10 20:57:07,686 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:07,717 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:07,719 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/count
2015-02-10 20:57:07,741 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:07,742 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:07,751 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:07,753 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:07,754 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:07,759 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting
2015-02-10 20:57:07,760 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:07,760 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] sink: Hfs["TextDelimited[['char', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby"]
2015-02-10 20:57:07,760 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] parallel execution is enabled: false
2015-02-10 20:57:07,760 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting jobs: 1
2015-02-10 20:57:07,760 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] allocating threads: 1
2015-02-10 20:57:07,761 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...mblyhelpersplatform/maxby
2015-02-10 20:57:07,762 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:07,763 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:07,789 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:07,807 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:07,823 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local822462648_0013
2015-02-10 20:57:07,841 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity822462648/.staging/job_local822462648_0013/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:07,841 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity822462648/.staging/job_local822462648_0013/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:07,891 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local822462648_0013/job_local822462648_0013.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:07,891 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local822462648_0013/job_local822462648_0013.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:07,893 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:07,893 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local822462648_0013
2015-02-10 20:57:07,895 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2015-02-10 20:57:07,895 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:07,895 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:07,898 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:07,898 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local822462648_0013_m_000000_0
2015-02-10 20:57:07,900 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:07,902 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:07,902 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6b6a16ae
2015-02-10 20:57:07,903 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:07,903 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:07,958 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:07,958 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:07,958 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:07,959 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:07,959 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:07,960 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:07,961 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:07,970 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:07,971 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'char']]
2015-02-10 20:57:07,971 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:07,974 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:07,975 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 117, total: 624, max: 672
2015-02-10 20:57:07,977 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:07,977 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 117, total: 624, max: 672
2015-02-10 20:57:07,978 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:07,978 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 117, total: 624, max: 672
2015-02-10 20:57:07,979 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:07,979 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 117, total: 624, max: 672
2015-02-10 20:57:07,981 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:07,982 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:07,982 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:07,982 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 135; bufvoid = 104857600
2015-02-10 20:57:07,983 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:07,987 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:07,989 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local822462648_0013_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:07,992 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:07,992 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local822462648_0013_m_000000_0' done.
2015-02-10 20:57:07,992 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local822462648_0013_m_000000_0
2015-02-10 20:57:07,993 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:07,994 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:07,994 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local822462648_0013_r_000000_0
2015-02-10 20:57:07,996 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:07,996 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@4d674592
2015-02-10 20:57:07,997 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:07,998 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local822462648_0013_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:08,001 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#11 about to shuffle output of map attempt_local822462648_0013_m_000000_0 decomp: 155 len: 159 to MEMORY
2015-02-10 20:57:08,001 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 155 bytes from map-output for attempt_local822462648_0013_m_000000_0
2015-02-10 20:57:08,001 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 155, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->155
2015-02-10 20:57:08,002 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:08,003 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:08,004 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:08,006 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:08,007 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2015-02-10 20:57:08,008 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 155 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:08,009 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 159 bytes from disk
2015-02-10 20:57:08,009 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:08,010 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:08,010 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2015-02-10 20:57:08,011 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:08,012 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:08,012 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:08,021 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'char']]
2015-02-10 20:57:08,022 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby"]
2015-02-10 20:57:08,033 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local822462648_0013_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:08,035 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:08,035 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local822462648_0013_r_000000_0 is allowed to commit now
2015-02-10 20:57:08,036 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local822462648_0013_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby/_temporary/0/task_local822462648_0013_r_000000
2015-02-10 20:57:08,038 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:08,038 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local822462648_0013_r_000000_0' done.
2015-02-10 20:57:08,039 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local822462648_0013_r_000000_0
2015-02-10 20:57:08,039 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:08,062 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby/_temporary
2015-02-10 20:57:08,097 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:08,139 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:08,141 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/maxby
2015-02-10 20:57:08,164 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:08,164 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:08,173 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:08,175 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:08,176 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:08,181 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] starting
2015-02-10 20:57:08,181 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:08,181 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] sink: Hfs["TextDelimited[['char', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby"]
2015-02-10 20:57:08,181 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] parallel execution is enabled: false
2015-02-10 20:57:08,182 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] starting jobs: 1
2015-02-10 20:57:08,182 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] allocating threads: 1
2015-02-10 20:57:08,182 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] starting step: (1/1) ...mblyhelpersplatform/minby
2015-02-10 20:57:08,183 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:08,184 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:08,208 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:08,226 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:08,241 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local2111875619_0014
2015-02-10 20:57:08,259 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity2111875619/.staging/job_local2111875619_0014/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:08,259 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity2111875619/.staging/job_local2111875619_0014/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:08,308 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local2111875619_0014/job_local2111875619_0014.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:08,308 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local2111875619_0014/job_local2111875619_0014.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:08,310 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:08,310 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] submitted hadoop job: job_local2111875619_0014
2015-02-10 20:57:08,311 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] tracking url: http://localhost:8080/
2015-02-10 20:57:08,310 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:08,311 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:08,315 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:08,315 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local2111875619_0014_m_000000_0
2015-02-10 20:57:08,317 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:08,319 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:08,319 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@14d6c410
2015-02-10 20:57:08,321 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:08,321 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:08,383 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:08,383 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:08,383 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:08,384 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:08,384 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:08,386 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:08,386 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:08,396 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:08,397 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(min)[by:[{1}:'char']]
2015-02-10 20:57:08,397 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:08,399 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:08,399 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 124, total: 674, max: 674
2015-02-10 20:57:08,400 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:08,401 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 124, total: 674, max: 674
2015-02-10 20:57:08,402 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:08,402 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 124, total: 674, max: 674
2015-02-10 20:57:08,403 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:08,403 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 124, total: 674, max: 674
2015-02-10 20:57:08,405 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:08,405 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:08,405 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:08,406 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 135; bufvoid = 104857600
2015-02-10 20:57:08,406 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:08,410 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:08,412 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local2111875619_0014_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:08,415 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:08,415 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2111875619_0014_m_000000_0' done.
2015-02-10 20:57:08,415 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local2111875619_0014_m_000000_0
2015-02-10 20:57:08,416 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:08,416 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:08,416 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local2111875619_0014_r_000000_0
2015-02-10 20:57:08,420 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:08,421 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@55aef45
2015-02-10 20:57:08,422 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=494855776, maxSingleShuffleLimit=123713944, mergeThreshold=326604832, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:08,423 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local2111875619_0014_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:08,425 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#12 about to shuffle output of map attempt_local2111875619_0014_m_000000_0 decomp: 155 len: 159 to MEMORY
2015-02-10 20:57:08,426 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 155 bytes from map-output for attempt_local2111875619_0014_m_000000_0
2015-02-10 20:57:08,426 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 155, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->155
2015-02-10 20:57:08,427 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:08,428 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:08,428 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:08,431 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:08,431 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2015-02-10 20:57:08,433 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 155 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:08,433 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 159 bytes from disk
2015-02-10 20:57:08,433 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:08,433 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:08,434 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2015-02-10 20:57:08,434 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:08,435 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:08,435 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:08,444 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(min)[by:[{1}:'char']]
2015-02-10 20:57:08,444 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby"]
2015-02-10 20:57:08,455 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local2111875619_0014_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:08,457 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:08,457 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local2111875619_0014_r_000000_0 is allowed to commit now
2015-02-10 20:57:08,458 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local2111875619_0014_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby/_temporary/0/task_local2111875619_0014_r_000000
2015-02-10 20:57:08,459 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:08,459 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2111875619_0014_r_000000_0' done.
2015-02-10 20:57:08,459 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local2111875619_0014_r_000000_0
2015-02-10 20:57:08,459 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:08,481 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby/_temporary
2015-02-10 20:57:08,516 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:08,552 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:08,554 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/minby
2015-02-10 20:57:08,587 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:08,588 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:08,596 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:08,598 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:08,598 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:08,605 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] starting
2015-02-10 20:57:08,605 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:08,605 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] sink: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum"]
2015-02-10 20:57:08,605 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] parallel execution is enabled: false
2015-02-10 20:57:08,606 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] starting jobs: 1
2015-02-10 20:57:08,606 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] allocating threads: 1
2015-02-10 20:57:08,606 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] starting step: (1/1) ...semblyhelpersplatform/sum
2015-02-10 20:57:08,607 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:08,608 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:08,632 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:08,650 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:08,665 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1443387747_0015
2015-02-10 20:57:08,683 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1443387747/.staging/job_local1443387747_0015/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:08,683 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1443387747/.staging/job_local1443387747_0015/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:08,730 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1443387747_0015/job_local1443387747_0015.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:08,730 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1443387747_0015/job_local1443387747_0015.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:08,731 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:08,731 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] submitted hadoop job: job_local1443387747_0015
2015-02-10 20:57:08,732 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] tracking url: http://localhost:8080/
2015-02-10 20:57:08,732 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:08,732 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:08,735 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:08,735 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1443387747_0015_m_000000_0
2015-02-10 20:57:08,737 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:08,739 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:08,740 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@13cb8654
2015-02-10 20:57:08,741 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:08,742 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:08,761 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:08,762 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:08,762 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:08,762 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:08,762 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:08,764 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:08,765 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:08,775 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:08,775 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum)[by:[{1}:'char']]
2015-02-10 20:57:08,776 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:08,777 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:08,777 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 128, total: 676, max: 676
2015-02-10 20:57:08,778 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:08,778 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 128, total: 676, max: 676
2015-02-10 20:57:08,779 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:08,779 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 128, total: 676, max: 676
2015-02-10 20:57:08,780 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:08,780 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 128, total: 676, max: 676
2015-02-10 20:57:08,781 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:08,781 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:08,781 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:08,782 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 20:57:08,782 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:08,785 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:08,787 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1443387747_0015_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:08,789 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:08,789 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1443387747_0015_m_000000_0' done.
2015-02-10 20:57:08,789 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1443387747_0015_m_000000_0
2015-02-10 20:57:08,790 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:08,790 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:08,791 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1443387747_0015_r_000000_0
2015-02-10 20:57:08,792 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:08,792 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@3f894e9c
2015-02-10 20:57:08,793 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=496599040, maxSingleShuffleLimit=124149760, mergeThreshold=327755392, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:08,794 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1443387747_0015_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:08,796 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#13 about to shuffle output of map attempt_local1443387747_0015_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 20:57:08,797 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local1443387747_0015_m_000000_0
2015-02-10 20:57:08,797 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 20:57:08,798 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:08,799 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:08,799 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:08,801 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:08,801 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:08,803 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:08,803 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 20:57:08,803 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:08,803 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:08,804 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:08,804 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:08,805 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:08,805 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:08,814 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(sum)[by:[{1}:'char']]
2015-02-10 20:57:08,815 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum"]
2015-02-10 20:57:08,826 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1443387747_0015_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:08,827 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:08,828 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1443387747_0015_r_000000_0 is allowed to commit now
2015-02-10 20:57:08,828 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1443387747_0015_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum/_temporary/0/task_local1443387747_0015_r_000000
2015-02-10 20:57:08,830 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:08,830 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1443387747_0015_r_000000_0' done.
2015-02-10 20:57:08,830 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1443387747_0015_r_000000_0
2015-02-10 20:57:08,830 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:08,850 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum/_temporary
2015-02-10 20:57:08,881 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:08,914 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:08,916 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/sum
2015-02-10 20:57:08,951 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:08,951 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:08,963 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:08,966 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:08,967 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:08,972 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs] starting
2015-02-10 20:57:08,973 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:57:08,973 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:08,974 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs] sink: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum"]
2015-02-10 20:57:08,975 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs] parallel execution is enabled: false
2015-02-10 20:57:08,975 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs] starting jobs: 1
2015-02-10 20:57:08,975 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs] allocating threads: 1
2015-02-10 20:57:08,976 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum-lhs+sum-rhs] starting step: (1/1) ...yhelpersplatform/mergesum
2015-02-10 20:57:08,977 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:08,978 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:09,002 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:09,003 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:09,021 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 20:57:09,037 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local387217018_0016
2015-02-10 20:57:09,058 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity387217018/.staging/job_local387217018_0016/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:09,058 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity387217018/.staging/job_local387217018_0016/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:09,111 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local387217018_0016/job_local387217018_0016.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:09,112 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local387217018_0016/job_local387217018_0016.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:09,113 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:09,114 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum-lhs+sum-rhs] submitted hadoop job: job_local387217018_0016
2015-02-10 20:57:09,114 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum-lhs+sum-rhs] tracking url: http://localhost:8080/
2015-02-10 20:57:09,114 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:09,115 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:09,118 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:09,118 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local387217018_0016_m_000000_0
2015-02-10 20:57:09,120 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:09,121 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:09,121 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@30307ae3
2015-02-10 20:57:09,122 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:09,122 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:09,145 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:09,146 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:09,146 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:09,146 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:09,146 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:09,147 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:09,147 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:09,159 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:09,160 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2015-02-10 20:57:09,160 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:09,162 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:09,163 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 162, total: 718, max: 718
2015-02-10 20:57:09,165 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:09,165 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 162, total: 718, max: 718
2015-02-10 20:57:09,167 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:09,167 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 162, total: 718, max: 718
2015-02-10 20:57:09,168 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:09,168 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 162, total: 718, max: 718
2015-02-10 20:57:09,171 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:09,171 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:09,171 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:09,171 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 20:57:09,171 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:09,175 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:09,176 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local387217018_0016_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:09,178 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:09,179 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local387217018_0016_m_000000_0' done.
2015-02-10 20:57:09,179 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local387217018_0016_m_000000_0
2015-02-10 20:57:09,179 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local387217018_0016_m_000001_0
2015-02-10 20:57:09,180 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:09,182 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt
2015-02-10 20:57:09,182 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6f8adce0
2015-02-10 20:57:09,183 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:09,184 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:09,215 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:09,215 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:09,216 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:09,216 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:09,216 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:09,217 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:09,217 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:09,228 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:57:09,228 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2015-02-10 20:57:09,229 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:09,233 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:09,233 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 213, total: 721, max: 721
2015-02-10 20:57:09,234 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:09,234 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 213, total: 721, max: 721
2015-02-10 20:57:09,235 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:09,235 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 213, total: 721, max: 721
2015-02-10 20:57:09,235 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:09,235 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 213, total: 721, max: 721
2015-02-10 20:57:09,237 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:09,237 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:09,237 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:09,237 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 20:57:09,237 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:09,240 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:09,242 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local387217018_0016_m_000001_0 is done. And is in the process of committing
2015-02-10 20:57:09,244 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 20:57:09,244 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local387217018_0016_m_000001_0' done.
2015-02-10 20:57:09,245 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local387217018_0016_m_000001_0
2015-02-10 20:57:09,245 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:09,245 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:09,245 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local387217018_0016_r_000000_0
2015-02-10 20:57:09,247 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:09,247 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5b4c1313
2015-02-10 20:57:09,252 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=529491552, maxSingleShuffleLimit=132372888, mergeThreshold=349464448, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:09,253 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local387217018_0016_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:09,255 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#14 about to shuffle output of map attempt_local387217018_0016_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 20:57:09,256 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local387217018_0016_m_000000_0
2015-02-10 20:57:09,256 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 20:57:09,258 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#14 about to shuffle output of map attempt_local387217018_0016_m_000001_0 decomp: 119 len: 123 to MEMORY
2015-02-10 20:57:09,258 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local387217018_0016_m_000001_0
2015-02-10 20:57:09,258 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 2, commitMemory -> 119, usedMemory ->238
2015-02-10 20:57:09,259 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:09,260 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:09,260 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:09,262 INFO mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2015-02-10 20:57:09,262 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 220 bytes
2015-02-10 20:57:09,264 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 238 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:09,264 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 240 bytes from disk
2015-02-10 20:57:09,265 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:09,265 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:09,265 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 227 bytes
2015-02-10 20:57:09,266 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:09,266 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:09,266 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:09,276 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2015-02-10 20:57:09,277 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum"]
2015-02-10 20:57:09,288 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local387217018_0016_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:09,289 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:09,289 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local387217018_0016_r_000000_0 is allowed to commit now
2015-02-10 20:57:09,290 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local387217018_0016_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum/_temporary/0/task_local387217018_0016_r_000000
2015-02-10 20:57:09,291 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:09,291 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local387217018_0016_r_000000_0' done.
2015-02-10 20:57:09,292 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local387217018_0016_r_000000_0
2015-02-10 20:57:09,292 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:09,319 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum/_temporary
2015-02-10 20:57:09,349 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:09,379 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:09,381 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/mergesum
2015-02-10 20:57:09,402 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:09,402 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:09,408 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:09,410 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:09,411 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:09,416 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 20:57:09,417 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:09,417 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] sink: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow"]
2015-02-10 20:57:09,417 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] parallel execution is enabled: false
2015-02-10 20:57:09,417 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting jobs: 1
2015-02-10 20:57:09,417 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] allocating threads: 1
2015-02-10 20:57:09,418 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...ersplatform/discardnarrow
2015-02-10 20:57:09,418 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:09,419 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:09,443 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:09,461 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:09,477 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1818488005_0017
2015-02-10 20:57:09,496 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1818488005/.staging/job_local1818488005_0017/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:09,496 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1818488005/.staging/job_local1818488005_0017/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:09,549 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1818488005_0017/job_local1818488005_0017.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:09,549 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1818488005_0017/job_local1818488005_0017.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:09,551 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:09,551 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1818488005_0017
2015-02-10 20:57:09,551 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:09,551 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 20:57:09,552 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:09,555 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:09,555 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1818488005_0017_m_000000_0
2015-02-10 20:57:09,557 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:09,559 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt
2015-02-10 20:57:09,559 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7e24f937
2015-02-10 20:57:09,561 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 20:57:09,569 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:09,569 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:09,575 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:09,576 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow"]
2015-02-10 20:57:09,579 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:09,579 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1818488005_0017_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:09,580 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:09,580 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1818488005_0017_m_000000_0 is allowed to commit now
2015-02-10 20:57:09,581 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1818488005_0017_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow/_temporary/0/task_local1818488005_0017_m_000000
2015-02-10 20:57:09,583 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt:0+19
2015-02-10 20:57:09,583 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1818488005_0017_m_000000_0' done.
2015-02-10 20:57:09,583 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1818488005_0017_m_000000_0
2015-02-10 20:57:09,583 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:09,596 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow/_temporary
2015-02-10 20:57:09,630 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:09,632 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/discardnarrow
2015-02-10 20:57:09,658 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:09,658 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:09,667 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:09,669 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:09,670 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:09,674 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 20:57:09,674 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:09,675 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] sink: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename"]
2015-02-10 20:57:09,675 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] parallel execution is enabled: false
2015-02-10 20:57:09,675 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting jobs: 1
2015-02-10 20:57:09,675 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] allocating threads: 1
2015-02-10 20:57:09,675 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...blyhelpersplatform/rename
2015-02-10 20:57:09,676 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:09,678 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:09,702 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:09,722 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:09,751 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1031387534_0018
2015-02-10 20:57:09,770 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1031387534/.staging/job_local1031387534_0018/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:09,770 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1031387534/.staging/job_local1031387534_0018/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:09,816 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1031387534_0018/job_local1031387534_0018.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:09,817 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1031387534_0018/job_local1031387534_0018.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:09,818 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:09,818 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1031387534_0018
2015-02-10 20:57:09,818 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:09,818 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 20:57:09,819 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:09,821 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:09,821 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1031387534_0018_m_000000_0
2015-02-10 20:57:09,823 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:09,824 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt
2015-02-10 20:57:09,824 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6204cfa9
2015-02-10 20:57:09,825 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 20:57:09,832 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:09,833 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:09,838 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:09,838 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename"]
2015-02-10 20:57:09,841 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:09,841 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1031387534_0018_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:09,843 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:09,843 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1031387534_0018_m_000000_0 is allowed to commit now
2015-02-10 20:57:09,843 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1031387534_0018_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename/_temporary/0/task_local1031387534_0018_m_000000
2015-02-10 20:57:09,844 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt:0+19
2015-02-10 20:57:09,845 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1031387534_0018_m_000000_0' done.
2015-02-10 20:57:09,845 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1031387534_0018_m_000000_0
2015-02-10 20:57:09,845 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:09,863 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename/_temporary
2015-02-10 20:57:09,892 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:09,894 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/rename
2015-02-10 20:57:09,917 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:09,918 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:09,932 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:09,935 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:09,935 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:09,942 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] starting
2015-02-10 20:57:09,942 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:57:09,942 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:09,943 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true"]
2015-02-10 20:57:09,943 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] parallel execution is enabled: false
2015-02-10 20:57:09,943 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] starting jobs: 1
2015-02-10 20:57:09,943 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] allocating threads: 1
2015-02-10 20:57:09,944 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] starting step: (1/1) ...splatform/multimerge+true
2015-02-10 20:57:09,944 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:09,945 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:09,969 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:09,970 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:09,989 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 20:57:10,005 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1435877350_0019
2015-02-10 20:57:10,022 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1435877350/.staging/job_local1435877350_0019/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:10,022 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1435877350/.staging/job_local1435877350_0019/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:10,068 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1435877350_0019/job_local1435877350_0019.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:10,069 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1435877350_0019/job_local1435877350_0019.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:10,070 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:10,070 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] submitted hadoop job: job_local1435877350_0019
2015-02-10 20:57:10,070 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] tracking url: http://localhost:8080/
2015-02-10 20:57:10,071 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:10,071 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:10,074 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:10,074 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1435877350_0019_m_000000_0
2015-02-10 20:57:10,076 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:10,077 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:10,078 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@52129dda
2015-02-10 20:57:10,079 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:10,079 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:10,104 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:10,104 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:10,104 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:10,105 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:10,105 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:10,108 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:10,108 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:10,122 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:10,122 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2015-02-10 20:57:10,123 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:10,125 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:10,125 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 161, total: 750, max: 750
2015-02-10 20:57:10,126 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:10,126 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 161, total: 750, max: 750
2015-02-10 20:57:10,127 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:10,127 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 161, total: 750, max: 750
2015-02-10 20:57:10,128 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:10,128 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 161, total: 750, max: 750
2015-02-10 20:57:10,129 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:10,129 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:10,129 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:10,129 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2015-02-10 20:57:10,130 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:10,133 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:10,135 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1435877350_0019_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:10,137 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:10,137 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1435877350_0019_m_000000_0' done.
2015-02-10 20:57:10,137 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1435877350_0019_m_000000_0
2015-02-10 20:57:10,137 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1435877350_0019_m_000001_0
2015-02-10 20:57:10,139 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:10,140 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt
2015-02-10 20:57:10,141 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6aed0f19
2015-02-10 20:57:10,143 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:10,143 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:10,168 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:10,168 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:10,169 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:10,169 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:10,169 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:10,171 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:10,171 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:10,185 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:57:10,185 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2015-02-10 20:57:10,185 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:10,189 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:10,190 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 750, max: 750
2015-02-10 20:57:10,191 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:10,191 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 750, max: 750
2015-02-10 20:57:10,192 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:10,192 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 750, max: 750
2015-02-10 20:57:10,193 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:10,193 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 750, max: 750
2015-02-10 20:57:10,195 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:10,195 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:10,195 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:10,195 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2015-02-10 20:57:10,195 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:10,198 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:10,200 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1435877350_0019_m_000001_0 is done. And is in the process of committing
2015-02-10 20:57:10,202 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 20:57:10,202 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1435877350_0019_m_000001_0' done.
2015-02-10 20:57:10,202 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1435877350_0019_m_000001_0
2015-02-10 20:57:10,203 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:10,203 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:10,204 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1435877350_0019_r_000000_0
2015-02-10 20:57:10,207 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:10,207 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@240fda77
2015-02-10 20:57:10,208 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551007040, maxSingleShuffleLimit=137751760, mergeThreshold=363664672, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:10,209 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1435877350_0019_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:10,211 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#15 about to shuffle output of map attempt_local1435877350_0019_m_000001_0 decomp: 236 len: 240 to MEMORY
2015-02-10 20:57:10,211 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local1435877350_0019_m_000001_0
2015-02-10 20:57:10,211 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->236
2015-02-10 20:57:10,214 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#15 about to shuffle output of map attempt_local1435877350_0019_m_000000_0 decomp: 236 len: 240 to MEMORY
2015-02-10 20:57:10,214 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local1435877350_0019_m_000000_0
2015-02-10 20:57:10,214 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 2, commitMemory -> 236, usedMemory ->472
2015-02-10 20:57:10,215 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:10,216 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:10,216 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:10,219 INFO mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2015-02-10 20:57:10,219 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 454 bytes
2015-02-10 20:57:10,221 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 472 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:10,222 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 474 bytes from disk
2015-02-10 20:57:10,222 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:10,222 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:10,222 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 461 bytes
2015-02-10 20:57:10,223 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:10,224 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:10,224 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:10,235 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(name)[by:[{1}:'char']]
2015-02-10 20:57:10,236 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true"]
2015-02-10 20:57:10,248 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1435877350_0019_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:10,250 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:10,250 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1435877350_0019_r_000000_0 is allowed to commit now
2015-02-10 20:57:10,250 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1435877350_0019_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true/_temporary/0/task_local1435877350_0019_r_000000
2015-02-10 20:57:10,252 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:10,252 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1435877350_0019_r_000000_0' done.
2015-02-10 20:57:10,252 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1435877350_0019_r_000000_0
2015-02-10 20:57:10,252 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:10,272 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true/_temporary
2015-02-10 20:57:10,301 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:10,333 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:10,336 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/multimerge+true
2015-02-10 20:57:10,362 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:10,362 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:10,386 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:10,389 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:10,395 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:10,401 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [second] starting
2015-02-10 20:57:10,401 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [second] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:10,402 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [second] sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount"]
2015-02-10 20:57:10,402 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [second] parallel execution is enabled: false
2015-02-10 20:57:10,402 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [second] starting jobs: 2
2015-02-10 20:57:10,402 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [second] allocating threads: 1
2015-02-10 20:57:10,403 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] starting step: (1/2)
2015-02-10 20:57:10,403 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:10,404 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:10,428 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:10,447 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:10,463 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1115004348_0020
2015-02-10 20:57:10,481 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1115004348/.staging/job_local1115004348_0020/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:10,482 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1115004348/.staging/job_local1115004348_0020/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:10,533 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1115004348_0020/job_local1115004348_0020.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:10,533 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1115004348_0020/job_local1115004348_0020.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:10,534 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:10,535 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:10,535 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] submitted hadoop job: job_local1115004348_0020
2015-02-10 20:57:10,535 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] tracking url: http://localhost:8080/
2015-02-10 20:57:10,535 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:10,537 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:10,538 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1115004348_0020_m_000000_0
2015-02-10 20:57:10,539 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:10,540 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:10,541 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4819d373
2015-02-10 20:57:10,542 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:10,542 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:10,567 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:10,568 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:10,568 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:10,568 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:10,568 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:10,569 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:10,569 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:10,579 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:10,579 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(first)[by:[{1}:'char']]
2015-02-10 20:57:10,580 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:10,584 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:10,584 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 750, max: 750
2015-02-10 20:57:10,585 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:10,585 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 750, max: 750
2015-02-10 20:57:10,586 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:10,586 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 750, max: 750
2015-02-10 20:57:10,587 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:10,587 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 750, max: 750
2015-02-10 20:57:10,588 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:10,588 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:10,589 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:10,589 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 20:57:10,589 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:10,592 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:10,594 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1115004348_0020_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:10,596 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:10,596 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1115004348_0020_m_000000_0' done.
2015-02-10 20:57:10,597 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1115004348_0020_m_000000_0
2015-02-10 20:57:10,597 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:10,597 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:10,598 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1115004348_0020_r_000000_0
2015-02-10 20:57:10,601 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:10,602 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5c635f9e
2015-02-10 20:57:10,603 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551098752, maxSingleShuffleLimit=137774688, mergeThreshold=363725184, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:10,604 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1115004348_0020_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:10,605 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#16 about to shuffle output of map attempt_local1115004348_0020_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 20:57:10,606 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local1115004348_0020_m_000000_0
2015-02-10 20:57:10,606 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 20:57:10,607 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:10,609 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:10,609 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:10,611 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:10,611 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:10,612 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:10,613 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 20:57:10,613 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:10,613 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:10,614 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:10,614 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:10,615 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:10,615 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:10,623 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(first)[by:[{1}:'char']]
2015-02-10 20:57:10,623 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][3708088709/first/]
2015-02-10 20:57:10,634 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1115004348_0020_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:10,635 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:10,635 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1115004348_0020_r_000000_0 is allowed to commit now
2015-02-10 20:57:10,636 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1115004348_0020_r_000000_0' to file:/tmp/hadoop-teamcity/3708088709_first_392E7FF95652417AA671A4FC02E969FF/_temporary/0/task_local1115004348_0020_r_000000
2015-02-10 20:57:10,637 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:10,637 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1115004348_0020_r_000000_0' done.
2015-02-10 20:57:10,638 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1115004348_0020_r_000000_0
2015-02-10 20:57:10,638 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:10,659 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] starting step: (2/2) ...form/samesourcemergecount
2015-02-10 20:57:10,659 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:10,660 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:10,697 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:10,821 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:10,831 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:10,832 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:10,837 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1035675950_0021
2015-02-10 20:57:10,854 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1035675950/.staging/job_local1035675950_0021/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:10,854 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1035675950/.staging/job_local1035675950_0021/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:10,900 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1035675950_0021/job_local1035675950_0021.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:10,901 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1035675950_0021/job_local1035675950_0021.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:10,902 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:10,902 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] submitted hadoop job: job_local1035675950_0021
2015-02-10 20:57:10,902 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] tracking url: http://localhost:8080/
2015-02-10 20:57:10,903 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:10,903 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:10,907 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:10,907 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1035675950_0021_m_000000_0
2015-02-10 20:57:10,909 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:10,910 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/3708088709_first_392E7FF95652417AA671A4FC02E969FF/part-00000
2015-02-10 20:57:10,910 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2d22860
2015-02-10 20:57:10,911 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:10,912 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:10,937 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:10,937 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:10,937 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:10,937 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:10,937 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:10,939 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:10,939 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:10,949 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][3708088709/first/]
2015-02-10 20:57:10,949 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(second)[by:[{1}:'char']]
2015-02-10 20:57:10,950 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:10,951 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:10,951 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:10,952 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:10,952 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:10,954 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:10,954 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:10,954 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:10,954 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 55; bufvoid = 104857600
2015-02-10 20:57:10,954 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:10,957 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:10,958 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1035675950_0021_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:10,960 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/3708088709_first_392E7FF95652417AA671A4FC02E969FF/part-00000:0+160
2015-02-10 20:57:10,961 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1035675950_0021_m_000000_0' done.
2015-02-10 20:57:10,961 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1035675950_0021_m_000000_0
2015-02-10 20:57:10,961 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:10,962 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:10,962 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1035675950_0021_r_000000_0
2015-02-10 20:57:10,963 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:10,963 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@66c5c0e5
2015-02-10 20:57:10,964 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551419904, maxSingleShuffleLimit=137854976, mergeThreshold=363937152, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:10,965 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1035675950_0021_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:10,973 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#17 about to shuffle output of map attempt_local1035675950_0021_m_000000_0 decomp: 67 len: 71 to MEMORY
2015-02-10 20:57:10,973 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 67 bytes from map-output for attempt_local1035675950_0021_m_000000_0
2015-02-10 20:57:10,973 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 67, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->67
2015-02-10 20:57:10,975 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:10,977 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:10,977 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:10,979 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:10,979 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 20:57:10,981 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 67 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:10,981 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 71 bytes from disk
2015-02-10 20:57:10,981 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:10,981 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:10,982 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 20:57:10,982 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:10,983 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:10,983 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:10,992 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(second)[by:[{1}:'char']]
2015-02-10 20:57:10,992 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount"]
2015-02-10 20:57:11,002 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1035675950_0021_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:11,004 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:11,004 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1035675950_0021_r_000000_0 is allowed to commit now
2015-02-10 20:57:11,005 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1035675950_0021_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount/_temporary/0/task_local1035675950_0021_r_000000
2015-02-10 20:57:11,006 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:11,007 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1035675950_0021_r_000000_0' done.
2015-02-10 20:57:11,007 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1035675950_0021_r_000000_0
2015-02-10 20:57:11,007 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:11,028 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount/_temporary
2015-02-10 20:57:11,060 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:11,092 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:11,094 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount
2015-02-10 20:57:11,115 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:11,115 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:11,124 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:11,126 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:11,126 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:11,131 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] starting
2015-02-10 20:57:11,132 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:11,132 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] sink: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average"]
2015-02-10 20:57:11,132 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] parallel execution is enabled: false
2015-02-10 20:57:11,133 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] starting jobs: 1
2015-02-10 20:57:11,133 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] allocating threads: 1
2015-02-10 20:57:11,134 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] starting step: (1/1) ...lyhelpersplatform/average
2015-02-10 20:57:11,135 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:11,136 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:11,160 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:11,178 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:11,194 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1241079119_0022
2015-02-10 20:57:11,212 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1241079119/.staging/job_local1241079119_0022/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:11,212 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1241079119/.staging/job_local1241079119_0022/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:11,259 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1241079119_0022/job_local1241079119_0022.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:11,260 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1241079119_0022/job_local1241079119_0022.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:11,261 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:11,262 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:11,262 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:11,262 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] submitted hadoop job: job_local1241079119_0022
2015-02-10 20:57:11,263 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] tracking url: http://localhost:8080/
2015-02-10 20:57:11,265 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:11,265 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1241079119_0022_m_000000_0
2015-02-10 20:57:11,266 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:11,268 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:11,268 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@48ca42a9
2015-02-10 20:57:11,269 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:11,269 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:11,294 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:11,295 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:11,295 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:11,295 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:11,295 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:11,296 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:11,296 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:11,303 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:11,303 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average)[by:[{1}:'char']]
2015-02-10 20:57:11,304 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:11,305 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:11,306 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,307 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:11,307 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,307 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:11,307 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,308 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:11,308 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,310 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:11,310 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:11,310 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:11,310 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2015-02-10 20:57:11,310 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:11,313 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:11,315 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1241079119_0022_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:11,317 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:11,317 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1241079119_0022_m_000000_0' done.
2015-02-10 20:57:11,317 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1241079119_0022_m_000000_0
2015-02-10 20:57:11,317 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:11,318 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:11,318 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1241079119_0022_r_000000_0
2015-02-10 20:57:11,320 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:11,320 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@2eb30073
2015-02-10 20:57:11,320 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551419904, maxSingleShuffleLimit=137854976, mergeThreshold=363937152, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:11,322 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1241079119_0022_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:11,323 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#18 about to shuffle output of map attempt_local1241079119_0022_m_000000_0 decomp: 200 len: 204 to MEMORY
2015-02-10 20:57:11,324 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local1241079119_0022_m_000000_0
2015-02-10 20:57:11,324 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->200
2015-02-10 20:57:11,325 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:11,326 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:11,326 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:11,328 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:11,328 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2015-02-10 20:57:11,329 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 200 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:11,330 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 204 bytes from disk
2015-02-10 20:57:11,330 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:11,330 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:11,330 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2015-02-10 20:57:11,331 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:11,331 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:11,332 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:11,339 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(average)[by:[{1}:'char']]
2015-02-10 20:57:11,339 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average"]
2015-02-10 20:57:11,349 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1241079119_0022_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:11,351 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:11,351 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1241079119_0022_r_000000_0 is allowed to commit now
2015-02-10 20:57:11,352 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1241079119_0022_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average/_temporary/0/task_local1241079119_0022_r_000000
2015-02-10 20:57:11,354 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:11,354 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1241079119_0022_r_000000_0' done.
2015-02-10 20:57:11,354 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1241079119_0022_r_000000_0
2015-02-10 20:57:11,355 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:11,375 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average/_temporary
2015-02-10 20:57:11,404 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:11,433 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:11,436 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/average
2015-02-10 20:57:11,459 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:11,460 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:11,467 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:11,469 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:11,470 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:11,476 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting
2015-02-10 20:57:11,476 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:11,476 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] sink: Hfs["TextDelimited[['char', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety"]
2015-02-10 20:57:11,476 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] parallel execution is enabled: false
2015-02-10 20:57:11,476 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting jobs: 1
2015-02-10 20:57:11,476 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] allocating threads: 1
2015-02-10 20:57:11,477 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...splatform/maxbynullsafety
2015-02-10 20:57:11,478 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:11,479 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:11,503 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:11,521 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:11,536 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local904167718_0023
2015-02-10 20:57:11,553 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity904167718/.staging/job_local904167718_0023/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:11,553 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity904167718/.staging/job_local904167718_0023/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:11,599 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local904167718_0023/job_local904167718_0023.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:11,599 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local904167718_0023/job_local904167718_0023.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:11,600 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:11,601 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local904167718_0023
2015-02-10 20:57:11,601 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:11,601 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2015-02-10 20:57:11,601 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:11,604 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:11,604 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local904167718_0023_m_000000_0
2015-02-10 20:57:11,606 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:11,607 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:11,607 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2fbe8a4b
2015-02-10 20:57:11,608 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:11,608 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:11,633 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:11,633 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:11,634 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:11,634 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:11,634 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:11,635 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:11,635 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:11,642 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:11,642 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'char']]
2015-02-10 20:57:11,643 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:11,644 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:11,644 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,645 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:11,646 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,646 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:11,646 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,647 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:11,647 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,649 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:11,649 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:11,649 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:11,649 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 98; bufvoid = 104857600
2015-02-10 20:57:11,649 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:11,652 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:11,654 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local904167718_0023_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:11,656 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:11,656 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local904167718_0023_m_000000_0' done.
2015-02-10 20:57:11,656 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local904167718_0023_m_000000_0
2015-02-10 20:57:11,657 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:11,657 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:11,657 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local904167718_0023_r_000000_0
2015-02-10 20:57:11,659 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:11,659 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7b8710ed
2015-02-10 20:57:11,660 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551695168, maxSingleShuffleLimit=137923792, mergeThreshold=364118816, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:11,660 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local904167718_0023_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:11,662 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#19 about to shuffle output of map attempt_local904167718_0023_m_000000_0 decomp: 118 len: 122 to MEMORY
2015-02-10 20:57:11,663 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 118 bytes from map-output for attempt_local904167718_0023_m_000000_0
2015-02-10 20:57:11,663 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 118, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->118
2015-02-10 20:57:11,664 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:11,665 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:11,665 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:11,667 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:11,667 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 109 bytes
2015-02-10 20:57:11,668 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 118 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:11,668 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 122 bytes from disk
2015-02-10 20:57:11,669 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:11,669 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:11,669 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 109 bytes
2015-02-10 20:57:11,670 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:11,670 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:11,670 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:11,678 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'char']]
2015-02-10 20:57:11,679 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety"]
2015-02-10 20:57:11,689 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local904167718_0023_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:11,690 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:11,691 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local904167718_0023_r_000000_0 is allowed to commit now
2015-02-10 20:57:11,691 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local904167718_0023_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety/_temporary/0/task_local904167718_0023_r_000000
2015-02-10 20:57:11,693 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:11,693 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local904167718_0023_r_000000_0' done.
2015-02-10 20:57:11,693 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local904167718_0023_r_000000_0
2015-02-10 20:57:11,693 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:11,715 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety/_temporary
2015-02-10 20:57:11,744 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:11,774 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:11,776 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety
2015-02-10 20:57:11,798 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:11,798 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:11,806 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:11,808 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:11,809 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:11,816 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] starting
2015-02-10 20:57:11,816 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:11,816 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] sink: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls"]
2015-02-10 20:57:11,816 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] parallel execution is enabled: false
2015-02-10 20:57:11,817 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] starting jobs: 1
2015-02-10 20:57:11,817 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] allocating threads: 1
2015-02-10 20:57:11,817 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] starting step: (1/1) ...yhelpersplatform/sumnulls
2015-02-10 20:57:11,818 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:11,819 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:11,844 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:11,862 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:11,877 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1857879576_0024
2015-02-10 20:57:11,894 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1857879576/.staging/job_local1857879576_0024/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:11,894 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1857879576/.staging/job_local1857879576_0024/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:11,941 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1857879576_0024/job_local1857879576_0024.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:11,941 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1857879576_0024/job_local1857879576_0024.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:11,942 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:11,942 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] submitted hadoop job: job_local1857879576_0024
2015-02-10 20:57:11,943 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] tracking url: http://localhost:8080/
2015-02-10 20:57:11,943 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:11,943 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:11,946 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:11,946 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1857879576_0024_m_000000_0
2015-02-10 20:57:11,948 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:11,949 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:11,949 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7b3825bf
2015-02-10 20:57:11,950 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:11,951 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:11,976 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:11,977 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:11,977 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:11,977 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:11,977 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:11,978 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:11,978 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:11,985 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:11,985 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum)[by:[{1}:'char']]
2015-02-10 20:57:11,986 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:11,995 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:11,995 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,996 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:11,996 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,997 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:11,997 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,997 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:11,998 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,999 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:11,999 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:11,999 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:11,999 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 96; bufvoid = 104857600
2015-02-10 20:57:12,000 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:12,003 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:12,004 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1857879576_0024_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:12,006 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:12,007 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1857879576_0024_m_000000_0' done.
2015-02-10 20:57:12,007 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1857879576_0024_m_000000_0
2015-02-10 20:57:12,007 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:12,008 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:12,008 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1857879576_0024_r_000000_0
2015-02-10 20:57:12,010 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:12,010 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@52ed6f9
2015-02-10 20:57:12,011 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551740992, maxSingleShuffleLimit=137935248, mergeThreshold=364149056, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:12,013 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1857879576_0024_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:12,015 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#20 about to shuffle output of map attempt_local1857879576_0024_m_000000_0 decomp: 116 len: 120 to MEMORY
2015-02-10 20:57:12,015 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 116 bytes from map-output for attempt_local1857879576_0024_m_000000_0
2015-02-10 20:57:12,016 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 116, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->116
2015-02-10 20:57:12,017 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:12,018 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:12,019 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:12,021 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:12,021 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 107 bytes
2015-02-10 20:57:12,023 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 116 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:12,024 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 120 bytes from disk
2015-02-10 20:57:12,024 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:12,024 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:12,025 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 107 bytes
2015-02-10 20:57:12,026 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:12,027 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:12,027 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:12,033 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(sum)[by:[{1}:'char']]
2015-02-10 20:57:12,034 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls"]
2015-02-10 20:57:12,043 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1857879576_0024_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:12,045 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:12,045 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1857879576_0024_r_000000_0 is allowed to commit now
2015-02-10 20:57:12,046 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1857879576_0024_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls/_temporary/0/task_local1857879576_0024_r_000000
2015-02-10 20:57:12,047 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:12,047 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1857879576_0024_r_000000_0' done.
2015-02-10 20:57:12,047 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1857879576_0024_r_000000_0
2015-02-10 20:57:12,047 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:12,066 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls/_temporary
2015-02-10 20:57:12,096 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:12,125 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:12,127 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/sumnulls
2015-02-10 20:57:12,146 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:12,147 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:12,152 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:12,154 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:12,154 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:12,158 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting
2015-02-10 20:57:12,159 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:12,159 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] sink: Hfs["TextDelimited[['count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall"]
2015-02-10 20:57:12,159 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] parallel execution is enabled: false
2015-02-10 20:57:12,159 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting jobs: 1
2015-02-10 20:57:12,159 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] allocating threads: 1
2015-02-10 20:57:12,160 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...yhelpersplatform/countall
2015-02-10 20:57:12,160 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:12,161 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:12,185 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:12,202 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:12,217 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local308241957_0025
2015-02-10 20:57:12,234 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity308241957/.staging/job_local308241957_0025/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:12,234 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity308241957/.staging/job_local308241957_0025/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:12,280 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local308241957_0025/job_local308241957_0025.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:12,280 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local308241957_0025/job_local308241957_0025.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:12,281 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:12,281 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local308241957_0025
2015-02-10 20:57:12,282 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-10 20:57:12,281 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:12,282 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:12,285 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:12,285 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local308241957_0025_m_000000_0
2015-02-10 20:57:12,288 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:12,289 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:12,289 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6b3a19e3
2015-02-10 20:57:12,291 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:12,291 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:12,317 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:12,317 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:12,317 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:12,318 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:12,318 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:12,319 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:12,319 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:12,326 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:12,327 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{?}:NONE]]
2015-02-10 20:57:12,327 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:12,330 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:12,330 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:12,330 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:12,331 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 4; bufvoid = 104857600
2015-02-10 20:57:12,331 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214396(104857584); length = 1/6553600
2015-02-10 20:57:12,333 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:12,335 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local308241957_0025_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:12,337 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:12,337 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local308241957_0025_m_000000_0' done.
2015-02-10 20:57:12,337 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local308241957_0025_m_000000_0
2015-02-10 20:57:12,337 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:12,338 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:12,338 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local308241957_0025_r_000000_0
2015-02-10 20:57:12,339 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:12,340 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5bf43093
2015-02-10 20:57:12,340 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552016256, maxSingleShuffleLimit=138004064, mergeThreshold=364330752, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:12,341 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local308241957_0025_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:12,343 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#21 about to shuffle output of map attempt_local308241957_0025_m_000000_0 decomp: 8 len: 12 to MEMORY
2015-02-10 20:57:12,343 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 8 bytes from map-output for attempt_local308241957_0025_m_000000_0
2015-02-10 20:57:12,344 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 8, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->8
2015-02-10 20:57:12,344 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:12,345 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:12,346 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:12,347 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:12,348 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 5 bytes
2015-02-10 20:57:12,349 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 8 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:12,349 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 12 bytes from disk
2015-02-10 20:57:12,349 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:12,349 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:12,350 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 5 bytes
2015-02-10 20:57:12,350 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:12,351 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:12,351 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:12,357 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{?}:NONE]]
2015-02-10 20:57:12,358 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall"]
2015-02-10 20:57:12,367 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local308241957_0025_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:12,368 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:12,368 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local308241957_0025_r_000000_0 is allowed to commit now
2015-02-10 20:57:12,369 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local308241957_0025_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall/_temporary/0/task_local308241957_0025_r_000000
2015-02-10 20:57:12,370 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:12,370 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local308241957_0025_r_000000_0' done.
2015-02-10 20:57:12,370 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local308241957_0025_r_000000_0
2015-02-10 20:57:12,370 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:12,387 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall/_temporary
2015-02-10 20:57:12,416 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:12,445 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:12,447 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/countall
2015-02-10 20:57:12,471 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:12,471 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:12,480 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:12,482 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:12,483 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:12,489 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] starting
2015-02-10 20:57:12,490 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:12,490 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average', 'average2', 'first']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi"]
2015-02-10 20:57:12,490 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] parallel execution is enabled: false
2015-02-10 20:57:12,490 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] starting jobs: 1
2015-02-10 20:57:12,490 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] allocating threads: 1
2015-02-10 20:57:12,491 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] starting step: (1/1) ...mblyhelpersplatform/multi
2015-02-10 20:57:12,491 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:12,492 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:12,518 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:12,536 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:12,551 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1483521596_0026
2015-02-10 20:57:12,569 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1483521596/.staging/job_local1483521596_0026/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:12,569 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1483521596/.staging/job_local1483521596_0026/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:12,617 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1483521596_0026/job_local1483521596_0026.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:12,617 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1483521596_0026/job_local1483521596_0026.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:12,618 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:12,619 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] submitted hadoop job: job_local1483521596_0026
2015-02-10 20:57:12,619 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:12,619 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] tracking url: http://localhost:8080/
2015-02-10 20:57:12,619 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:12,622 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:12,622 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1483521596_0026_m_000000_0
2015-02-10 20:57:12,623 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:12,625 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:12,625 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@34985fc6
2015-02-10 20:57:12,626 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:12,626 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:12,652 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:12,652 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:12,652 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:12,652 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:12,653 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:12,654 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:12,655 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:12,664 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:12,664 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2015-02-10 20:57:12,664 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:12,666 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:12,666 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 752, max: 752
2015-02-10 20:57:12,667 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:12,668 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 752, max: 752
2015-02-10 20:57:12,668 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:12,669 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 752, max: 752
2015-02-10 20:57:12,671 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:12,671 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 752, max: 752
2015-02-10 20:57:12,673 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:12,673 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:12,673 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:12,673 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 432; bufvoid = 104857600
2015-02-10 20:57:12,673 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:12,676 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:12,678 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1483521596_0026_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:12,680 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:12,680 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1483521596_0026_m_000000_0' done.
2015-02-10 20:57:12,680 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1483521596_0026_m_000000_0
2015-02-10 20:57:12,681 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:12,681 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:12,681 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1483521596_0026_r_000000_0
2015-02-10 20:57:12,686 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:12,686 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@170e3e70
2015-02-10 20:57:12,687 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552108032, maxSingleShuffleLimit=138027008, mergeThreshold=364391328, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:12,688 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1483521596_0026_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:12,691 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#22 about to shuffle output of map attempt_local1483521596_0026_m_000000_0 decomp: 452 len: 456 to MEMORY
2015-02-10 20:57:12,691 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 452 bytes from map-output for attempt_local1483521596_0026_m_000000_0
2015-02-10 20:57:12,692 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 452, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->452
2015-02-10 20:57:12,693 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:12,694 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:12,694 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:12,696 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:12,696 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 436 bytes
2015-02-10 20:57:12,698 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 452 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:12,698 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 456 bytes from disk
2015-02-10 20:57:12,699 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:12,699 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:12,700 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 436 bytes
2015-02-10 20:57:12,700 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:12,701 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:12,701 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:12,710 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(name)[by:[{1}:'char']]
2015-02-10 20:57:12,711 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average', 'average2', 'first']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi"]
2015-02-10 20:57:12,722 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1483521596_0026_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:12,723 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:12,723 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1483521596_0026_r_000000_0 is allowed to commit now
2015-02-10 20:57:12,724 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1483521596_0026_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi/_temporary/0/task_local1483521596_0026_r_000000
2015-02-10 20:57:12,725 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:12,725 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1483521596_0026_r_000000_0' done.
2015-02-10 20:57:12,725 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1483521596_0026_r_000000_0
2015-02-10 20:57:12,726 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:12,743 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi/_temporary
2015-02-10 20:57:12,772 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:12,804 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:12,806 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/multi
2015-02-10 20:57:12,825 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:12,825 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:12,830 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:12,831 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:12,832 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:12,837 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 20:57:12,838 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:12,838 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] sink: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow"]
2015-02-10 20:57:12,839 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] parallel execution is enabled: false
2015-02-10 20:57:12,840 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting jobs: 1
2015-02-10 20:57:12,840 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] allocating threads: 1
2015-02-10 20:57:12,841 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...persplatform/retainnarrow
2015-02-10 20:57:12,842 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:12,843 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:12,867 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:12,884 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:12,899 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1309572996_0027
2015-02-10 20:57:12,917 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1309572996/.staging/job_local1309572996_0027/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:12,917 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1309572996/.staging/job_local1309572996_0027/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:12,962 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1309572996_0027/job_local1309572996_0027.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:12,963 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1309572996_0027/job_local1309572996_0027.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:12,964 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:12,965 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:12,965 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:12,966 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1309572996_0027
2015-02-10 20:57:12,966 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 20:57:12,968 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:12,968 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1309572996_0027_m_000000_0
2015-02-10 20:57:12,970 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:12,971 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt
2015-02-10 20:57:12,971 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@647be610
2015-02-10 20:57:12,972 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 20:57:12,979 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:12,979 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:12,983 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:12,983 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow"]
2015-02-10 20:57:12,986 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:12,986 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1309572996_0027_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:12,987 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:12,988 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1309572996_0027_m_000000_0 is allowed to commit now
2015-02-10 20:57:12,988 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1309572996_0027_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow/_temporary/0/task_local1309572996_0027_m_000000
2015-02-10 20:57:12,989 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt:0+19
2015-02-10 20:57:12,990 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1309572996_0027_m_000000_0' done.
2015-02-10 20:57:12,990 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1309572996_0027_m_000000_0
2015-02-10 20:57:12,990 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:13,010 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow/_temporary
2015-02-10 20:57:13,039 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:13,041 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/retainnarrow
2015-02-10 20:57:13,060 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:13,061 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:13,066 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:13,068 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:13,069 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:13,073 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 20:57:13,073 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:13,073 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] sink: Hfs["TextLine[['item']->['char', 'item']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow"]
2015-02-10 20:57:13,073 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] parallel execution is enabled: false
2015-02-10 20:57:13,073 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting jobs: 1
2015-02-10 20:57:13,074 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] allocating threads: 1
2015-02-10 20:57:13,074 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...persplatform/renamenarrow
2015-02-10 20:57:13,075 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:13,077 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:13,100 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:13,118 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:13,138 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1439863029_0028
2015-02-10 20:57:13,160 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1439863029/.staging/job_local1439863029_0028/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:13,160 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1439863029/.staging/job_local1439863029_0028/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:13,219 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1439863029_0028/job_local1439863029_0028.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:13,219 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1439863029_0028/job_local1439863029_0028.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:13,221 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:13,221 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1439863029_0028
2015-02-10 20:57:13,221 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:13,221 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 20:57:13,222 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:13,225 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:13,225 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1439863029_0028_m_000000_0
2015-02-10 20:57:13,226 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:13,228 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt
2015-02-10 20:57:13,228 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5e0469bc
2015-02-10 20:57:13,230 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 20:57:13,237 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:13,237 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:13,242 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:13,242 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['item']->['char', 'item']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow"]
2015-02-10 20:57:13,244 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:13,245 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1439863029_0028_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:13,246 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:13,246 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1439863029_0028_m_000000_0 is allowed to commit now
2015-02-10 20:57:13,247 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1439863029_0028_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow/_temporary/0/task_local1439863029_0028_m_000000
2015-02-10 20:57:13,248 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt:0+19
2015-02-10 20:57:13,248 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1439863029_0028_m_000000_0' done.
2015-02-10 20:57:13,249 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1439863029_0028_m_000000_0
2015-02-10 20:57:13,249 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:13,265 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow/_temporary
2015-02-10 20:57:13,297 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:13,299 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/renamenarrow
2015-02-10 20:57:13,325 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:13,326 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:13,335 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:13,337 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:13,338 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:13,343 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] starting
2015-02-10 20:57:13,343 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:13,344 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] sink: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull"]
2015-02-10 20:57:13,344 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] parallel execution is enabled: false
2015-02-10 20:57:13,344 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] starting jobs: 1
2015-02-10 20:57:13,344 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] allocating threads: 1
2015-02-10 20:57:13,345 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] starting step: (1/1) ...lpersplatform/averagenull
2015-02-10 20:57:13,346 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:13,347 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:13,371 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:13,390 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:13,409 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1491983852_0029
2015-02-10 20:57:13,436 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1491983852/.staging/job_local1491983852_0029/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:13,436 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1491983852/.staging/job_local1491983852_0029/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:13,482 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1491983852_0029/job_local1491983852_0029.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:13,483 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1491983852_0029/job_local1491983852_0029.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:13,484 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:13,484 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:13,484 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:13,484 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] submitted hadoop job: job_local1491983852_0029
2015-02-10 20:57:13,485 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] tracking url: http://localhost:8080/
2015-02-10 20:57:13,487 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:13,487 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1491983852_0029_m_000000_0
2015-02-10 20:57:13,488 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:13,489 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:13,490 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4876d42
2015-02-10 20:57:13,491 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:13,491 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:13,508 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:13,508 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:13,508 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:13,509 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:13,509 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:13,510 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:13,510 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:13,517 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:13,518 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average)[by:[{1}:'char']]
2015-02-10 20:57:13,518 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:13,522 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:13,522 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 228, total: 752, max: 752
2015-02-10 20:57:13,523 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:13,523 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 228, total: 752, max: 752
2015-02-10 20:57:13,526 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:13,526 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 228, total: 752, max: 752
2015-02-10 20:57:13,527 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:13,527 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 228, total: 752, max: 752
2015-02-10 20:57:13,528 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:13,528 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:13,528 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:13,528 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2015-02-10 20:57:13,529 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:13,531 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:13,533 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1491983852_0029_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:13,535 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:13,535 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1491983852_0029_m_000000_0' done.
2015-02-10 20:57:13,535 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1491983852_0029_m_000000_0
2015-02-10 20:57:13,535 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:13,536 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:13,536 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1491983852_0029_r_000000_0
2015-02-10 20:57:13,538 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:13,538 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@b920c51
2015-02-10 20:57:13,538 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552337408, maxSingleShuffleLimit=138084352, mergeThreshold=364542688, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:13,539 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1491983852_0029_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:13,541 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#23 about to shuffle output of map attempt_local1491983852_0029_m_000000_0 decomp: 200 len: 204 to MEMORY
2015-02-10 20:57:13,541 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local1491983852_0029_m_000000_0
2015-02-10 20:57:13,541 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->200
2015-02-10 20:57:13,542 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:13,543 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:13,543 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:13,545 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:13,545 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2015-02-10 20:57:13,546 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 200 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:13,547 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 204 bytes from disk
2015-02-10 20:57:13,547 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:13,547 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:13,547 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2015-02-10 20:57:13,548 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:13,548 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:13,548 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:13,556 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(average)[by:[{1}:'char']]
2015-02-10 20:57:13,556 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull"]
2015-02-10 20:57:13,567 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1491983852_0029_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:13,568 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:13,568 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1491983852_0029_r_000000_0 is allowed to commit now
2015-02-10 20:57:13,569 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1491983852_0029_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull/_temporary/0/task_local1491983852_0029_r_000000
2015-02-10 20:57:13,570 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:13,570 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1491983852_0029_r_000000_0' done.
2015-02-10 20:57:13,570 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1491983852_0029_r_000000_0
2015-02-10 20:57:13,570 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:13,591 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull/_temporary
2015-02-10 20:57:13,621 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:13,653 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:13,655 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/averagenull
2015-02-10 20:57:13,672 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:13,673 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:13,677 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:13,679 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:13,679 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:13,683 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] starting
2015-02-10 20:57:13,683 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:13,684 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] sink: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce"]
2015-02-10 20:57:13,684 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] parallel execution is enabled: false
2015-02-10 20:57:13,684 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] starting jobs: 1
2015-02-10 20:57:13,685 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] allocating threads: 1
2015-02-10 20:57:13,686 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] starting step: (1/1) ...blyhelpersplatform/coerce
2015-02-10 20:57:13,687 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:13,688 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:13,712 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:13,730 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:13,745 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local314091455_0030
2015-02-10 20:57:13,762 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity314091455/.staging/job_local314091455_0030/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:13,762 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity314091455/.staging/job_local314091455_0030/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:13,816 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local314091455_0030/job_local314091455_0030.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:13,817 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local314091455_0030/job_local314091455_0030.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:13,818 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:13,818 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:13,819 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:13,818 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] submitted hadoop job: job_local314091455_0030
2015-02-10 20:57:13,819 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] tracking url: http://localhost:8080/
2015-02-10 20:57:13,821 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:13,821 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local314091455_0030_m_000000_0
2015-02-10 20:57:13,822 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:13,824 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt
2015-02-10 20:57:13,824 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3512106c
2015-02-10 20:57:13,825 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 20:57:13,831 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:13,832 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:13,835 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:13,836 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce"]
2015-02-10 20:57:13,838 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:13,838 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local314091455_0030_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:13,840 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:13,840 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local314091455_0030_m_000000_0 is allowed to commit now
2015-02-10 20:57:13,841 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local314091455_0030_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce/_temporary/0/task_local314091455_0030_m_000000
2015-02-10 20:57:13,842 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt:0+19
2015-02-10 20:57:13,842 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local314091455_0030_m_000000_0' done.
2015-02-10 20:57:13,842 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local314091455_0030_m_000000_0
2015-02-10 20:57:13,842 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:13,852 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce/_temporary
2015-02-10 20:57:13,883 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:13,885 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/coerce
2015-02-10 20:57:13,907 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:13,907 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:13,914 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:13,916 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:13,916 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:13,921 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] starting
2015-02-10 20:57:13,921 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] source: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross-rev.txt"]
2015-02-10 20:57:13,921 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] sink: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator"]
2015-02-10 20:57:13,922 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] parallel execution is enabled: false
2015-02-10 20:57:13,922 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] starting jobs: 1
2015-02-10 20:57:13,922 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] allocating threads: 1
2015-02-10 20:57:13,922 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] starting step: (1/1) ...tnfieldswithoutcomparator
2015-02-10 20:57:13,923 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:13,924 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:13,946 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:13,965 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:13,982 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local715569249_0031
2015-02-10 20:57:13,998 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity715569249/.staging/job_local715569249_0031/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:13,998 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity715569249/.staging/job_local715569249_0031/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:14,046 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local715569249_0031/job_local715569249_0031.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:14,047 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local715569249_0031/job_local715569249_0031.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:14,048 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:14,049 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:14,049 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] submitted hadoop job: job_local715569249_0031
2015-02-10 20:57:14,049 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] tracking url: http://localhost:8080/
2015-02-10 20:57:14,049 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:14,051 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:14,052 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local715569249_0031_m_000000_0
2015-02-10 20:57:14,053 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:14,055 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross-rev.txt
2015-02-10 20:57:14,055 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6600167a
2015-02-10 20:57:14,056 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:14,056 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:14,087 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:14,088 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:14,088 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:14,088 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:14,088 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:14,089 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:14,089 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:14,095 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross-rev.txt"]
2015-02-10 20:57:14,095 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(first)[by:[{1}:'num']]
2015-02-10 20:57:14,096 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:14,098 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:14,098 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 241, total: 752, max: 752
2015-02-10 20:57:14,099 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:14,099 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 241, total: 752, max: 752
2015-02-10 20:57:14,101 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:14,101 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:14,102 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:14,102 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 105; bufvoid = 104857600
2015-02-10 20:57:14,102 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:14,104 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:14,106 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local715569249_0031_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:14,109 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross-rev.txt:0+221
2015-02-10 20:57:14,109 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local715569249_0031_m_000000_0' done.
2015-02-10 20:57:14,109 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local715569249_0031_m_000000_0
2015-02-10 20:57:14,109 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:14,110 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:14,110 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local715569249_0031_r_000000_0
2015-02-10 20:57:14,112 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:14,112 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@573c3ba2
2015-02-10 20:57:14,113 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552383296, maxSingleShuffleLimit=138095824, mergeThreshold=364572992, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:14,114 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local715569249_0031_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:14,117 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#24 about to shuffle output of map attempt_local715569249_0031_m_000000_0 decomp: 117 len: 121 to MEMORY
2015-02-10 20:57:14,117 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 117 bytes from map-output for attempt_local715569249_0031_m_000000_0
2015-02-10 20:57:14,117 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 117, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->117
2015-02-10 20:57:14,118 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:14,119 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:14,119 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:14,122 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:14,122 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 108 bytes
2015-02-10 20:57:14,124 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 117 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:14,124 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 121 bytes from disk
2015-02-10 20:57:14,125 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:14,125 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:14,126 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 108 bytes
2015-02-10 20:57:14,127 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:14,127 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:14,128 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:14,133 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(first)[by:[{1}:'num']]
2015-02-10 20:57:14,133 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator"]
2015-02-10 20:57:14,143 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local715569249_0031_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:14,144 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:14,144 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local715569249_0031_r_000000_0 is allowed to commit now
2015-02-10 20:57:14,145 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local715569249_0031_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator/_temporary/0/task_local715569249_0031_r_000000
2015-02-10 20:57:14,146 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:14,147 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local715569249_0031_r_000000_0' done.
2015-02-10 20:57:14,147 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local715569249_0031_r_000000_0
2015-02-10 20:57:14,147 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:14,166 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator/_temporary
2015-02-10 20:57:14,194 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:14,196 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator
2015-02-10 20:57:14,219 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:14,220 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:14,245 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:14,247 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:14,269 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:14,274 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...] starting
2015-02-10 20:57:14,274 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:14,274 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...] sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway"]
2015-02-10 20:57:14,275 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...] parallel execution is enabled: false
2015-02-10 20:57:14,275 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...] starting jobs: 5
2015-02-10 20:57:14,275 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...] allocating threads: 1
2015-02-10 20:57:14,275 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (1/5)
2015-02-10 20:57:14,276 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:14,278 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:14,301 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:14,319 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:14,334 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local2013585284_0032
2015-02-10 20:57:14,350 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity2013585284/.staging/job_local2013585284_0032/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:14,350 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity2013585284/.staging/job_local2013585284_0032/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:14,395 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local2013585284_0032/job_local2013585284_0032.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:14,395 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local2013585284_0032/job_local2013585284_0032.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:14,396 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:14,397 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local2013585284_0032
2015-02-10 20:57:14,397 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:14,397 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-10 20:57:14,397 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:14,399 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:14,399 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local2013585284_0032_m_000000_0
2015-02-10 20:57:14,401 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:14,402 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:14,402 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@499aed36
2015-02-10 20:57:14,403 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:14,403 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:14,429 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:14,429 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:14,429 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:14,429 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:14,429 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:14,431 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:14,431 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:14,439 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:14,439 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(source)[by:[{1}:'char']]
2015-02-10 20:57:14,439 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:14,440 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:14,441 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 241, total: 752, max: 752
2015-02-10 20:57:14,441 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:14,442 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 241, total: 752, max: 752
2015-02-10 20:57:14,442 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:14,442 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 241, total: 752, max: 752
2015-02-10 20:57:14,443 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:14,443 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 241, total: 752, max: 752
2015-02-10 20:57:14,444 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:14,444 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:14,444 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:14,445 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 20:57:14,445 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:14,448 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:14,449 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local2013585284_0032_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:14,451 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:14,451 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2013585284_0032_m_000000_0' done.
2015-02-10 20:57:14,451 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local2013585284_0032_m_000000_0
2015-02-10 20:57:14,452 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:14,452 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:14,452 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local2013585284_0032_r_000000_0
2015-02-10 20:57:14,456 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:14,456 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5482fb8f
2015-02-10 20:57:14,457 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552566784, maxSingleShuffleLimit=138141696, mergeThreshold=364694080, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:14,458 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local2013585284_0032_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:14,460 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#25 about to shuffle output of map attempt_local2013585284_0032_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 20:57:14,460 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local2013585284_0032_m_000000_0
2015-02-10 20:57:14,460 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 20:57:14,461 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:14,462 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:14,462 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:14,464 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:14,465 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:14,466 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:14,466 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 20:57:14,467 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:14,467 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:14,468 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:14,468 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:14,469 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:14,469 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:14,474 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(source)[by:[{1}:'char']]
2015-02-10 20:57:14,474 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][4782051044/source/]
2015-02-10 20:57:14,483 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local2013585284_0032_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:14,484 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:14,485 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local2013585284_0032_r_000000_0 is allowed to commit now
2015-02-10 20:57:14,485 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local2013585284_0032_r_000000_0' to file:/tmp/hadoop-teamcity/4782051044_source_FD52B13C054D45C88E578801EBB52573/_temporary/0/task_local2013585284_0032_r_000000
2015-02-10 20:57:14,486 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:14,486 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2013585284_0032_r_000000_0' done.
2015-02-10 20:57:14,487 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local2013585284_0032_r_000000_0
2015-02-10 20:57:14,487 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:14,507 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (3/5)
2015-02-10 20:57:14,507 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:14,508 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:14,543 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:14,560 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:14,571 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:14,571 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:14,576 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1416992176_0033
2015-02-10 20:57:14,593 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1416992176/.staging/job_local1416992176_0033/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:14,593 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1416992176/.staging/job_local1416992176_0033/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:14,640 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1416992176_0033/job_local1416992176_0033.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:14,641 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1416992176_0033/job_local1416992176_0033.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:14,642 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:14,642 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:14,643 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:14,643 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1416992176_0033
2015-02-10 20:57:14,643 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-10 20:57:14,646 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:14,646 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1416992176_0033_m_000000_0
2015-02-10 20:57:14,647 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:14,648 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/4782051044_source_FD52B13C054D45C88E578801EBB52573/part-00000
2015-02-10 20:57:14,649 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5ee55d17
2015-02-10 20:57:14,650 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:14,650 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:14,687 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:14,687 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:14,687 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:14,688 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:14,688 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:14,689 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:14,689 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:14,695 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][4782051044/source/]
2015-02-10 20:57:14,695 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-lhs)[by:[{1}:'char']]
2015-02-10 20:57:14,695 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:14,696 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:14,696 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 96, total: 752, max: 752
2015-02-10 20:57:14,697 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:14,697 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 96, total: 752, max: 752
2015-02-10 20:57:14,698 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:14,698 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:14,698 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:14,699 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 55; bufvoid = 104857600
2015-02-10 20:57:14,958 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:14,961 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:14,963 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1416992176_0033_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:14,965 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/4782051044_source_FD52B13C054D45C88E578801EBB52573/part-00000:0+160
2015-02-10 20:57:14,965 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1416992176_0033_m_000000_0' done.
2015-02-10 20:57:14,965 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1416992176_0033_m_000000_0
2015-02-10 20:57:14,965 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:14,966 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:14,966 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1416992176_0033_r_000000_0
2015-02-10 20:57:14,967 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:14,968 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@6cd8739a
2015-02-10 20:57:14,968 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:14,969 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1416992176_0033_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:14,970 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#26 about to shuffle output of map attempt_local1416992176_0033_m_000000_0 decomp: 67 len: 71 to MEMORY
2015-02-10 20:57:14,971 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 67 bytes from map-output for attempt_local1416992176_0033_m_000000_0
2015-02-10 20:57:14,971 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 67, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->67
2015-02-10 20:57:14,972 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:14,973 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:14,973 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:14,975 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:14,975 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 20:57:14,976 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 67 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:14,976 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 71 bytes from disk
2015-02-10 20:57:14,976 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:14,977 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:14,977 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 20:57:14,977 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:14,978 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:14,978 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:14,983 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count-lhs)[by:[{1}:'char']]
2015-02-10 20:57:14,983 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][662217636/count-lhs/]
2015-02-10 20:57:14,992 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1416992176_0033_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:14,993 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:14,993 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1416992176_0033_r_000000_0 is allowed to commit now
2015-02-10 20:57:14,994 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1416992176_0033_r_000000_0' to file:/tmp/hadoop-teamcity/662217636_count_lhs_4FB7D5D97F0A45A2B1A5E4C0AC039061/_temporary/0/task_local1416992176_0033_r_000000
2015-02-10 20:57:14,995 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:14,995 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1416992176_0033_r_000000_0' done.
2015-02-10 20:57:14,995 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1416992176_0033_r_000000_0
2015-02-10 20:57:14,995 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:15,013 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (4/5)
2015-02-10 20:57:15,014 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:15,015 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:15,050 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:15,068 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:15,078 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:15,078 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:15,084 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local892925828_0034
2015-02-10 20:57:15,100 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity892925828/.staging/job_local892925828_0034/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:15,100 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity892925828/.staging/job_local892925828_0034/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:15,145 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local892925828_0034/job_local892925828_0034.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:15,145 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local892925828_0034/job_local892925828_0034.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:15,146 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:15,147 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local892925828_0034
2015-02-10 20:57:15,147 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-10 20:57:15,147 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:15,148 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:15,150 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:15,151 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local892925828_0034_m_000000_0
2015-02-10 20:57:15,152 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:15,154 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/4782051044_source_FD52B13C054D45C88E578801EBB52573/part-00000
2015-02-10 20:57:15,154 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@439ca904
2015-02-10 20:57:15,156 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:15,156 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:15,175 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:15,175 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:15,175 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:15,175 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:15,175 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:15,177 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:15,177 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:15,185 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][4782051044/source/]
2015-02-10 20:57:15,185 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-rhs)[by:[{1}:'char']]
2015-02-10 20:57:15,185 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:15,187 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:15,187 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 172, total: 427, max: 672
2015-02-10 20:57:15,187 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 63% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:15,187 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:15,188 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 172, total: 427, max: 672
2015-02-10 20:57:15,188 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 63% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:15,189 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:15,189 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:15,189 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:15,189 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 55; bufvoid = 104857600
2015-02-10 20:57:15,189 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:15,192 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:15,194 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local892925828_0034_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:15,196 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/4782051044_source_FD52B13C054D45C88E578801EBB52573/part-00000:0+160
2015-02-10 20:57:15,196 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local892925828_0034_m_000000_0' done.
2015-02-10 20:57:15,196 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local892925828_0034_m_000000_0
2015-02-10 20:57:15,196 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:15,197 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:15,197 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local892925828_0034_r_000000_0
2015-02-10 20:57:15,199 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:15,199 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@6d0cad7b
2015-02-10 20:57:15,200 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:15,200 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local892925828_0034_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:15,204 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#27 about to shuffle output of map attempt_local892925828_0034_m_000000_0 decomp: 67 len: 71 to MEMORY
2015-02-10 20:57:15,204 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 67 bytes from map-output for attempt_local892925828_0034_m_000000_0
2015-02-10 20:57:15,205 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 67, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->67
2015-02-10 20:57:15,205 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:15,206 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:15,207 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:15,209 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:15,209 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 20:57:15,210 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 67 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:15,211 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 71 bytes from disk
2015-02-10 20:57:15,211 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:15,211 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:15,211 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 20:57:15,212 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:15,212 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:15,212 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:15,218 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count-rhs)[by:[{1}:'char']]
2015-02-10 20:57:15,218 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][511480183/count-rhs/]
2015-02-10 20:57:15,227 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local892925828_0034_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:15,228 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:15,228 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local892925828_0034_r_000000_0 is allowed to commit now
2015-02-10 20:57:15,229 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local892925828_0034_r_000000_0' to file:/tmp/hadoop-teamcity/511480183_count_rhs_2F723E24535B40D9B8C83AABD3957A53/_temporary/0/task_local892925828_0034_r_000000
2015-02-10 20:57:15,230 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:15,230 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local892925828_0034_r_000000_0' done.
2015-02-10 20:57:15,230 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local892925828_0034_r_000000_0
2015-02-10 20:57:15,230 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:15,243 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (2/5)
2015-02-10 20:57:15,243 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:15,244 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:15,279 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:15,292 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:15,310 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 20:57:15,320 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:15,320 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:15,326 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1058891099_0035
2015-02-10 20:57:15,343 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1058891099/.staging/job_local1058891099_0035/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:15,343 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1058891099/.staging/job_local1058891099_0035/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:15,389 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1058891099_0035/job_local1058891099_0035.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:15,390 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1058891099_0035/job_local1058891099_0035.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:15,391 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:15,392 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1058891099_0035
2015-02-10 20:57:15,392 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:15,392 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-10 20:57:15,392 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:15,395 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:15,395 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1058891099_0035_m_000000_0
2015-02-10 20:57:15,397 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:15,398 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/4782051044_source_FD52B13C054D45C88E578801EBB52573/part-00000
2015-02-10 20:57:15,398 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3ff9ddb5
2015-02-10 20:57:15,400 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:15,400 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:15,423 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:15,423 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:15,423 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:15,423 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:15,423 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:15,426 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:15,426 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:15,433 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][4782051044/source/]
2015-02-10 20:57:15,433 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 20:57:15,436 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:15,436 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:15,436 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:15,436 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 65; bufvoid = 104857600
2015-02-10 20:57:15,436 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:15,439 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:15,441 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1058891099_0035_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:15,442 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/4782051044_source_FD52B13C054D45C88E578801EBB52573/part-00000:0+160
2015-02-10 20:57:15,442 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1058891099_0035_m_000000_0' done.
2015-02-10 20:57:15,443 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1058891099_0035_m_000000_0
2015-02-10 20:57:15,443 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1058891099_0035_m_000001_0
2015-02-10 20:57:15,444 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:15,445 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/511480183_count_rhs_2F723E24535B40D9B8C83AABD3957A53/part-00000
2015-02-10 20:57:15,445 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7809ead8
2015-02-10 20:57:15,447 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:15,447 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:15,464 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:15,464 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:15,465 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:15,465 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:15,465 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:15,466 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:15,466 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:15,474 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][511480183/count-rhs/]
2015-02-10 20:57:15,475 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 20:57:15,476 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:15,477 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:15,477 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:15,477 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 65; bufvoid = 104857600
2015-02-10 20:57:15,477 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:15,479 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:15,481 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1058891099_0035_m_000001_0 is done. And is in the process of committing
2015-02-10 20:57:15,483 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/511480183_count_rhs_2F723E24535B40D9B8C83AABD3957A53/part-00000:0+160
2015-02-10 20:57:15,483 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1058891099_0035_m_000001_0' done.
2015-02-10 20:57:15,483 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1058891099_0035_m_000001_0
2015-02-10 20:57:15,483 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:15,484 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:15,484 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1058891099_0035_r_000000_0
2015-02-10 20:57:15,485 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:15,485 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@42793673
2015-02-10 20:57:15,486 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:15,487 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1058891099_0035_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:15,488 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#28 about to shuffle output of map attempt_local1058891099_0035_m_000001_0 decomp: 77 len: 81 to MEMORY
2015-02-10 20:57:15,489 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 77 bytes from map-output for attempt_local1058891099_0035_m_000001_0
2015-02-10 20:57:15,489 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 77, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->77
2015-02-10 20:57:15,491 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#28 about to shuffle output of map attempt_local1058891099_0035_m_000000_0 decomp: 77 len: 81 to MEMORY
2015-02-10 20:57:15,492 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 77 bytes from map-output for attempt_local1058891099_0035_m_000000_0
2015-02-10 20:57:15,492 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 77, inMemoryMapOutputs.size() -> 2, commitMemory -> 77, usedMemory ->154
2015-02-10 20:57:15,493 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:15,495 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:15,496 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:15,498 INFO mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2015-02-10 20:57:15,498 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 134 bytes
2015-02-10 20:57:15,500 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 154 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:15,500 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 156 bytes from disk
2015-02-10 20:57:15,501 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:15,501 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:15,501 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 142 bytes
2015-02-10 20:57:15,502 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:15,503 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:15,503 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:15,508 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 20:57:15,509 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'num', 'char2', 'count']]"][5223282736/source_count-rhs/]
2015-02-10 20:57:15,523 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(105)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-10 20:57:15,523 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(110)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-10 20:57:15,530 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1058891099_0035_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:15,531 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:15,531 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1058891099_0035_r_000000_0 is allowed to commit now
2015-02-10 20:57:15,532 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1058891099_0035_r_000000_0' to file:/tmp/hadoop-teamcity/5223282736_source_count_rhs_0EFDC0F8252B47848ABB1557CC1685D9/_temporary/0/task_local1058891099_0035_r_000000
2015-02-10 20:57:15,533 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:15,533 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1058891099_0035_r_000000_0' done.
2015-02-10 20:57:15,533 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1058891099_0035_r_000000_0
2015-02-10 20:57:15,534 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:15,552 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (5/5) ...m/samesourcemergethreeway
2015-02-10 20:57:15,552 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:15,553 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:15,588 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:15,602 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:15,620 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 20:57:15,629 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:15,629 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:15,635 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1092641491_0036
2015-02-10 20:57:15,651 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1092641491/.staging/job_local1092641491_0036/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:15,651 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1092641491/.staging/job_local1092641491_0036/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:15,704 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1092641491_0036/job_local1092641491_0036.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:15,704 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1092641491_0036/job_local1092641491_0036.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:15,706 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:15,706 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1092641491_0036
2015-02-10 20:57:15,706 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-10 20:57:15,707 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:15,707 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:15,834 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:15,834 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1092641491_0036_m_000000_0
2015-02-10 20:57:15,836 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:15,838 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/5223282736_source_count_rhs_0EFDC0F8252B47848ABB1557CC1685D9/part-00000
2015-02-10 20:57:15,838 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1f78a4e8
2015-02-10 20:57:15,840 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:15,840 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:15,858 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:15,858 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:15,858 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:15,858 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:15,859 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:15,861 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:15,861 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:15,868 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'num', 'char2', 'count']]"][5223282736/source_count-rhs/]
2015-02-10 20:57:15,869 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2015-02-10 20:57:15,870 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:15,871 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:15,871 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:15,871 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 105; bufvoid = 104857600
2015-02-10 20:57:15,871 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:15,874 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:15,875 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1092641491_0036_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:15,877 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/5223282736_source_count_rhs_0EFDC0F8252B47848ABB1557CC1685D9/part-00000:0+200
2015-02-10 20:57:15,877 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1092641491_0036_m_000000_0' done.
2015-02-10 20:57:15,877 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1092641491_0036_m_000000_0
2015-02-10 20:57:15,877 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1092641491_0036_m_000001_0
2015-02-10 20:57:15,878 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:15,879 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/662217636_count_lhs_4FB7D5D97F0A45A2B1A5E4C0AC039061/part-00000
2015-02-10 20:57:15,880 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3ccc2187
2015-02-10 20:57:15,881 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:15,881 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:15,899 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:15,899 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:15,899 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:15,899 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:15,899 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:15,900 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:15,901 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:15,911 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][662217636/count-lhs/]
2015-02-10 20:57:15,911 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2015-02-10 20:57:15,914 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:15,914 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:15,915 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:15,915 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 65; bufvoid = 104857600
2015-02-10 20:57:15,915 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:15,918 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:15,920 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1092641491_0036_m_000001_0 is done. And is in the process of committing
2015-02-10 20:57:15,921 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/662217636_count_lhs_4FB7D5D97F0A45A2B1A5E4C0AC039061/part-00000:0+160
2015-02-10 20:57:15,922 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1092641491_0036_m_000001_0' done.
2015-02-10 20:57:15,922 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1092641491_0036_m_000001_0
2015-02-10 20:57:15,922 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:15,923 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:15,923 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1092641491_0036_r_000000_0
2015-02-10 20:57:15,925 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:15,925 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5f92e49f
2015-02-10 20:57:15,926 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:15,926 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1092641491_0036_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:15,928 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#29 about to shuffle output of map attempt_local1092641491_0036_m_000001_0 decomp: 77 len: 81 to MEMORY
2015-02-10 20:57:15,928 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 77 bytes from map-output for attempt_local1092641491_0036_m_000001_0
2015-02-10 20:57:15,928 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 77, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->77
2015-02-10 20:57:15,930 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#29 about to shuffle output of map attempt_local1092641491_0036_m_000000_0 decomp: 117 len: 121 to MEMORY
2015-02-10 20:57:15,931 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 117 bytes from map-output for attempt_local1092641491_0036_m_000000_0
2015-02-10 20:57:15,931 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 117, inMemoryMapOutputs.size() -> 2, commitMemory -> 77, usedMemory ->194
2015-02-10 20:57:15,931 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:15,932 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:15,932 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:15,934 INFO mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2015-02-10 20:57:15,934 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 174 bytes
2015-02-10 20:57:15,936 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 194 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:15,936 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 196 bytes from disk
2015-02-10 20:57:15,936 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:15,936 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:15,937 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 182 bytes
2015-02-10 20:57:15,937 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:15,938 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:15,938 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:15,944 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2015-02-10 20:57:15,945 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway"]
2015-02-10 20:57:15,953 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(105)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-10 20:57:15,953 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(110)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-10 20:57:15,955 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1092641491_0036_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:15,956 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:15,957 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1092641491_0036_r_000000_0 is allowed to commit now
2015-02-10 20:57:15,957 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1092641491_0036_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway/_temporary/0/task_local1092641491_0036_r_000000
2015-02-10 20:57:15,958 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:15,958 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1092641491_0036_r_000000_0' done.
2015-02-10 20:57:15,958 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1092641491_0036_r_000000_0
2015-02-10 20:57:15,959 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:15,978 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway/_temporary
2015-02-10 20:57:16,006 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:16,035 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:16,036 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway
2015-02-10 20:57:16,058 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:16,059 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:16,066 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:16,069 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:16,069 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:16,077 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs] starting
2015-02-10 20:57:16,077 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:57:16,077 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:16,078 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs] sink: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic"]
2015-02-10 20:57:16,078 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs] parallel execution is enabled: false
2015-02-10 20:57:16,078 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs] starting jobs: 1
2015-02-10 20:57:16,078 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs] allocating threads: 1
2015-02-10 20:57:16,079 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lhs+rhs] starting step: (1/1) ...quemerge-nondeterministic
2015-02-10 20:57:16,079 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:16,080 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:16,109 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:16,109 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:16,129 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 20:57:16,144 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local533979663_0037
2015-02-10 20:57:16,162 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity533979663/.staging/job_local533979663_0037/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:16,163 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity533979663/.staging/job_local533979663_0037/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:16,208 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local533979663_0037/job_local533979663_0037.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:16,208 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local533979663_0037/job_local533979663_0037.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:16,209 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:16,210 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lhs+rhs] submitted hadoop job: job_local533979663_0037
2015-02-10 20:57:16,210 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:16,210 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:16,210 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lhs+rhs] tracking url: http://localhost:8080/
2015-02-10 20:57:16,212 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:16,213 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local533979663_0037_m_000000_0
2015-02-10 20:57:16,214 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:16,215 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:16,215 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3026a5f8
2015-02-10 20:57:16,216 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:16,217 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:16,234 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:16,234 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:16,234 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:16,234 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:16,234 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:16,235 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:16,235 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:16,241 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:16,241 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2015-02-10 20:57:16,247 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:16,247 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:16,247 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:16,247 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-10 20:57:16,248 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:16,250 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:16,252 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local533979663_0037_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:16,253 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:16,253 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local533979663_0037_m_000000_0' done.
2015-02-10 20:57:16,254 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local533979663_0037_m_000000_0
2015-02-10 20:57:16,254 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local533979663_0037_m_000001_0
2015-02-10 20:57:16,255 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:16,256 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt
2015-02-10 20:57:16,256 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@27d2b7b3
2015-02-10 20:57:16,257 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:16,258 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:16,275 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:16,275 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:16,275 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:16,275 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:16,275 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:16,279 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:16,279 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:16,284 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:57:16,284 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2015-02-10 20:57:16,287 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:16,287 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:16,287 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:16,287 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-10 20:57:16,288 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:16,290 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:16,292 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local533979663_0037_m_000001_0 is done. And is in the process of committing
2015-02-10 20:57:16,293 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 20:57:16,293 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local533979663_0037_m_000001_0' done.
2015-02-10 20:57:16,294 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local533979663_0037_m_000001_0
2015-02-10 20:57:16,294 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:16,294 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:16,294 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local533979663_0037_r_000000_0
2015-02-10 20:57:16,296 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:16,296 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@1d3313b1
2015-02-10 20:57:16,297 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:16,299 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local533979663_0037_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:16,300 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#30 about to shuffle output of map attempt_local533979663_0037_m_000001_0 decomp: 87 len: 91 to MEMORY
2015-02-10 20:57:16,301 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local533979663_0037_m_000001_0
2015-02-10 20:57:16,301 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2015-02-10 20:57:16,303 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#30 about to shuffle output of map attempt_local533979663_0037_m_000000_0 decomp: 87 len: 91 to MEMORY
2015-02-10 20:57:16,303 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local533979663_0037_m_000000_0
2015-02-10 20:57:16,303 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 2, commitMemory -> 87, usedMemory ->174
2015-02-10 20:57:16,304 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:16,305 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:16,305 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:16,307 INFO mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2015-02-10 20:57:16,307 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 156 bytes
2015-02-10 20:57:16,308 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 174 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:16,309 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 176 bytes from disk
2015-02-10 20:57:16,309 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:16,309 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:16,309 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 163 bytes
2015-02-10 20:57:16,310 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:16,310 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:16,310 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:16,316 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2015-02-10 20:57:16,316 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic"]
2015-02-10 20:57:16,326 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local533979663_0037_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:16,327 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:16,327 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local533979663_0037_r_000000_0 is allowed to commit now
2015-02-10 20:57:16,328 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local533979663_0037_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic/_temporary/0/task_local533979663_0037_r_000000
2015-02-10 20:57:16,329 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:16,329 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local533979663_0037_r_000000_0' done.
2015-02-10 20:57:16,329 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local533979663_0037_r_000000_0
2015-02-10 20:57:16,329 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:16,358 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic/_temporary
2015-02-10 20:57:16,387 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:16,389 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic
2015-02-10 20:57:16,407 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:16,408 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:16,414 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:16,416 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:16,417 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:16,421 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 20:57:16,421 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:16,422 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] sink: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique"]
2015-02-10 20:57:16,422 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] parallel execution is enabled: false
2015-02-10 20:57:16,422 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting jobs: 1
2015-02-10 20:57:16,423 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] allocating threads: 1
2015-02-10 20:57:16,425 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...blyhelpersplatform/unique
2015-02-10 20:57:16,425 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:16,427 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:16,449 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:16,467 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:16,482 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1602462514_0038
2015-02-10 20:57:16,499 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1602462514/.staging/job_local1602462514_0038/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:16,499 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1602462514/.staging/job_local1602462514_0038/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:16,545 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1602462514_0038/job_local1602462514_0038.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:16,545 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1602462514_0038/job_local1602462514_0038.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:16,547 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:16,547 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:16,547 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:16,547 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1602462514_0038
2015-02-10 20:57:16,548 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 20:57:16,550 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:16,550 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1602462514_0038_m_000000_0
2015-02-10 20:57:16,551 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:16,553 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:16,553 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5db533b0
2015-02-10 20:57:16,554 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:16,554 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:16,571 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:16,571 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:16,572 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:16,572 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:16,572 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:16,573 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:16,573 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:16,578 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:16,578 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(shape)[by:[{1}:'num']]
2015-02-10 20:57:16,581 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:16,581 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:16,581 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:16,581 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-10 20:57:16,581 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:16,584 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:16,587 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1602462514_0038_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:16,589 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:16,589 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1602462514_0038_m_000000_0' done.
2015-02-10 20:57:16,589 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1602462514_0038_m_000000_0
2015-02-10 20:57:16,589 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:16,590 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:16,590 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1602462514_0038_r_000000_0
2015-02-10 20:57:16,591 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:16,592 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@1d489e0e
2015-02-10 20:57:16,592 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:16,593 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1602462514_0038_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:16,594 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#31 about to shuffle output of map attempt_local1602462514_0038_m_000000_0 decomp: 87 len: 91 to MEMORY
2015-02-10 20:57:16,595 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local1602462514_0038_m_000000_0
2015-02-10 20:57:16,595 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2015-02-10 20:57:16,596 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:16,597 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:16,597 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:16,599 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:16,599 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 20:57:16,600 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 87 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:16,601 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 91 bytes from disk
2015-02-10 20:57:16,601 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:16,601 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:16,601 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 20:57:16,602 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:16,602 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:16,602 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:16,606 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(shape)[by:[{1}:'num']]
2015-02-10 20:57:16,606 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique"]
2015-02-10 20:57:16,615 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1602462514_0038_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:16,616 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:16,616 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1602462514_0038_r_000000_0 is allowed to commit now
2015-02-10 20:57:16,617 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1602462514_0038_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique/_temporary/0/task_local1602462514_0038_r_000000
2015-02-10 20:57:16,618 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:16,618 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1602462514_0038_r_000000_0' done.
2015-02-10 20:57:16,618 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1602462514_0038_r_000000_0
2015-02-10 20:57:16,619 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:16,634 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique/_temporary
2015-02-10 20:57:16,662 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:16,664 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/unique
2015-02-10 20:57:16,684 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:16,684 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:16,690 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:16,692 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:16,693 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:16,697 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting
2015-02-10 20:57:16,697 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:16,697 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] sink: Hfs["TextDelimited[['notnull', 'null']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull"]
2015-02-10 20:57:16,697 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] parallel execution is enabled: false
2015-02-10 20:57:16,698 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting jobs: 1
2015-02-10 20:57:16,698 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] allocating threads: 1
2015-02-10 20:57:16,698 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...platform/countnullnotnull
2015-02-10 20:57:16,699 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:16,700 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:16,723 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:16,741 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:16,755 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local195034871_0039
2015-02-10 20:57:16,773 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity195034871/.staging/job_local195034871_0039/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:16,773 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity195034871/.staging/job_local195034871_0039/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:16,821 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local195034871_0039/job_local195034871_0039.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:57:16,821 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local195034871_0039/job_local195034871_0039.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:57:16,823 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:16,823 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local195034871_0039
2015-02-10 20:57:16,824 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:16,824 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-10 20:57:16,824 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:16,826 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:16,827 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local195034871_0039_m_000000_0
2015-02-10 20:57:16,828 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:16,829 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:16,829 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@78f278dd
2015-02-10 20:57:16,830 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:16,830 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:16,853 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:16,853 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:16,853 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:16,853 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:16,854 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:16,855 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:16,855 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:16,863 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:16,863 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{?}:NONE]]
2015-02-10 20:57:16,864 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:16,870 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:57:16,870 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:16,870 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:16,871 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 6; bufvoid = 104857600
2015-02-10 20:57:16,871 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214396(104857584); length = 1/6553600
2015-02-10 20:57:16,873 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:16,875 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local195034871_0039_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:16,877 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:16,877 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local195034871_0039_m_000000_0' done.
2015-02-10 20:57:16,878 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local195034871_0039_m_000000_0
2015-02-10 20:57:16,878 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:16,878 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:16,879 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local195034871_0039_r_000000_0
2015-02-10 20:57:16,880 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:16,881 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@c7d9406
2015-02-10 20:57:16,881 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:16,882 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local195034871_0039_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:16,884 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#32 about to shuffle output of map attempt_local195034871_0039_m_000000_0 decomp: 10 len: 14 to MEMORY
2015-02-10 20:57:16,884 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 10 bytes from map-output for attempt_local195034871_0039_m_000000_0
2015-02-10 20:57:16,884 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 10, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->10
2015-02-10 20:57:16,885 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:16,886 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:16,886 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:16,888 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:16,888 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 7 bytes
2015-02-10 20:57:16,890 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 10 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:16,890 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 14 bytes from disk
2015-02-10 20:57:16,891 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:16,891 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:16,891 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 7 bytes
2015-02-10 20:57:16,892 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:16,893 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:16,893 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:16,901 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{?}:NONE]]
2015-02-10 20:57:16,901 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['notnull', 'null']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull"]
2015-02-10 20:57:16,909 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local195034871_0039_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:16,910 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:16,911 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local195034871_0039_r_000000_0 is allowed to commit now
2015-02-10 20:57:16,911 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local195034871_0039_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull/_temporary/0/task_local195034871_0039_r_000000
2015-02-10 20:57:16,912 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:16,912 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local195034871_0039_r_000000_0' done.
2015-02-10 20:57:16,912 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local195034871_0039_r_000000_0
2015-02-10 20:57:16,913 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:16,932 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull/_temporary
2015-02-10 20:57:16,961 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:16,989 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:16,991 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/countnullnotnull