Class cascading.pipe.assembly.AssemblyHelpersPlatformTest

33

tests

0

failures

19.705s

duration

100%

successful

Tests

Test Duration Result
testAverageBy 0.365s passed
testAverageByNull 0.350s passed
testAverageMerge 0.918s passed
testCoerce 0.231s passed
testCoerceFields 0.293s passed
testCount 0.462s passed
testCountAll 0.352s passed
testCountCount 1.045s passed
testCountMerge 0.794s passed
testCountNullNotNull 0.345s passed
testDiscardNarrow 0.259s passed
testFirstBy 0.553s passed
testFirstByWithoutComparator 0.313s passed
testMaxBy 0.489s passed
testMaxByNullSafety 0.361s passed
testMaxByString 0.518s passed
testMinBy 0.374s passed
testMinByNullSafety 0.754s passed
testMinByString 0.704s passed
testParallelAggregates 0.362s passed
testParallelAggregatesMerge 3.568s passed
testParallelAggregatesPriorMerge 0.470s passed
testRenameAll 0.690s passed
testRenameNamed 0.292s passed
testRenameNarrow 0.248s passed
testRetainNarrow 0.257s passed
testSameSourceMerge 0.663s passed
testSameSourceMergeThreeWay 1.671s passed
testSumBy 0.436s passed
testSumByNulls 0.358s passed
testSumMerge 0.519s passed
testUnique 0.310s passed
testUniqueMerge 0.381s passed

Standard output

2015-02-10 22:48:41,284 INFO  platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(187)) - found 1 test platforms from classpath
2015-02-10 22:48:41,352 INFO  platform.PlatformRunner (PlatformRunner.java:addPlatform(237)) - installing platform: hadoop2-mr1
2015-02-10 22:48:41,352 INFO  platform.PlatformRunner (PlatformRunner.java:addPlatform(238)) - running test: cascading.pipe.assembly.AssemblyHelpersPlatformTest
2015-02-10 22:48:41,400 INFO  hadoop2.Hadoop2MR1Platform (Hadoop2MR1Platform.java:setUp(105)) - not using cluster
2015-02-10 22:48:42,045 WARN  util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-02-10 22:48:42,650 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:42,653 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:42,665 INFO  property.AppProps (AppProps.java:getAppID(169)) - using app.id: 3C067E13787C4D43B366892043BA8958
2015-02-10 22:48:42,764 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:42,806 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:42,809 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:42,960 INFO  util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 2.6.3
2015-02-10 22:48:42,962 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name] starting
2015-02-10 22:48:42,963 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:48:42,964 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:42,964 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false"]
2015-02-10 22:48:42,966 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  parallel execution is enabled: false
2015-02-10 22:48:42,966 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  starting jobs: 1
2015-02-10 22:48:42,966 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  allocating threads: 1
2015-02-10 22:48:42,968 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] starting step: (1/1) ...platform/multimerge+false
2015-02-10 22:48:42,992 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - session.id is deprecated. Instead, use dfs.metrics.session-id
2015-02-10 22:48:42,993 INFO  jvm.JvmMetrics (JvmMetrics.java:init(76)) - Initializing JVM Metrics with processName=JobTracker, sessionId=
2015-02-10 22:48:43,021 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:43,346 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:43,358 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:43,389 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 22:48:43,550 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local782323639_0001
2015-02-10 22:48:43,592 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity782323639/.staging/job_local782323639_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:43,605 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity782323639/.staging/job_local782323639_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:43,795 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local782323639_0001/job_local782323639_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:43,803 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local782323639_0001/job_local782323639_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:43,810 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:43,812 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:43,815 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] submitted hadoop job: job_local782323639_0001
2015-02-10 22:48:43,816 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] tracking url: http://localhost:8080/
2015-02-10 22:48:43,816 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:43,888 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:43,891 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local782323639_0001_m_000000_0
2015-02-10 22:48:43,936 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:43,946 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:43,946 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@565539d8
2015-02-10 22:48:43,955 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:43,962 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:44,124 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:44,124 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:44,124 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:44,124 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:44,125 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:44,154 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:44,155 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:44,218 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
2015-02-10 22:48:44,226 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:44,227 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2015-02-10 22:48:44,230 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:44,245 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:44,246 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 88, total: 202, max: 672
2015-02-10 22:48:44,246 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 30% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:44,247 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:44,248 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 88, total: 202, max: 672
2015-02-10 22:48:44,248 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 30% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:44,249 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:44,249 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 88, total: 202, max: 672
2015-02-10 22:48:44,249 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 30% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:44,250 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:44,250 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 88, total: 202, max: 672
2015-02-10 22:48:44,251 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 30% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:44,255 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:44,255 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:44,255 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:44,255 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2015-02-10 22:48:44,256 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:44,270 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:44,274 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local782323639_0001_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:44,286 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:44,288 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local782323639_0001_m_000000_0' done.
2015-02-10 22:48:44,288 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local782323639_0001_m_000000_0
2015-02-10 22:48:44,288 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local782323639_0001_m_000001_0
2015-02-10 22:48:44,291 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:44,293 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt
2015-02-10 22:48:44,293 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@c58f769
2015-02-10 22:48:44,295 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:44,295 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:44,449 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:44,450 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:44,450 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:44,450 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:44,450 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:44,454 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:44,454 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:44,481 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:48:44,481 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2015-02-10 22:48:44,482 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:44,578 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:44,578 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 83, total: 302, max: 672
2015-02-10 22:48:44,578 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 44% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:44,580 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:44,580 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 83, total: 302, max: 672
2015-02-10 22:48:44,580 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 44% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:44,582 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:44,582 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 83, total: 302, max: 672
2015-02-10 22:48:44,582 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 44% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:44,583 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:44,583 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 83, total: 302, max: 672
2015-02-10 22:48:44,583 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 44% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:44,585 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:44,585 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:44,586 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:44,586 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2015-02-10 22:48:44,586 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:44,592 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:44,595 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local782323639_0001_m_000001_0 is done. And is in the process of committing
2015-02-10 22:48:44,598 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 22:48:44,598 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local782323639_0001_m_000001_0' done.
2015-02-10 22:48:44,599 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local782323639_0001_m_000001_0
2015-02-10 22:48:44,599 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:44,602 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:44,602 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local782323639_0001_r_000000_0
2015-02-10 22:48:44,610 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:44,613 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@1b435686
2015-02-10 22:48:44,627 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:44,631 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local782323639_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:44,661 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#1 about to shuffle output of map attempt_local782323639_0001_m_000001_0 decomp: 236 len: 240 to MEMORY
2015-02-10 22:48:44,666 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local782323639_0001_m_000001_0
2015-02-10 22:48:44,708 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->236
2015-02-10 22:48:44,712 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#1 about to shuffle output of map attempt_local782323639_0001_m_000000_0 decomp: 236 len: 240 to MEMORY
2015-02-10 22:48:44,713 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local782323639_0001_m_000000_0
2015-02-10 22:48:44,713 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 2, commitMemory -> 236, usedMemory ->472
2015-02-10 22:48:44,714 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:44,715 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:44,716 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:44,728 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 2 sorted segments
2015-02-10 22:48:44,729 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 2 segments left of total size: 454 bytes
2015-02-10 22:48:44,732 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 472 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:44,733 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 474 bytes from disk
2015-02-10 22:48:44,733 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:44,734 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:44,734 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 461 bytes
2015-02-10 22:48:44,735 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:44,738 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:44,738 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:44,765 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2015-02-10 22:48:44,766 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false"]
2015-02-10 22:48:44,795 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local782323639_0001_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:44,797 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:44,798 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local782323639_0001_r_000000_0 is allowed to commit now
2015-02-10 22:48:44,799 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local782323639_0001_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false/_temporary/0/task_local782323639_0001_r_000000
2015-02-10 22:48:44,800 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:44,801 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local782323639_0001_r_000000_0' done.
2015-02-10 22:48:44,801 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local782323639_0001_r_000000_0
2015-02-10 22:48:44,801 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:44,867 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false/_temporary
2015-02-10 22:48:44,916 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:44,961 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:44,965 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/multimerge+false
2015-02-10 22:48:45,003 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:45,004 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:45,022 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:45,028 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:45,029 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:45,047 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting
2015-02-10 22:48:45,047 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:45,048 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  sink: Hfs["TextDelimited[['num', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring"]
2015-02-10 22:48:45,048 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  parallel execution is enabled: false
2015-02-10 22:48:45,048 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  starting jobs: 1
2015-02-10 22:48:45,050 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  allocating threads: 1
2015-02-10 22:48:45,051 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...lpersplatform/minbystring
2015-02-10 22:48:45,052 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:45,054 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:45,087 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:45,114 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:45,150 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local975825011_0002
2015-02-10 22:48:45,177 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity975825011/.staging/job_local975825011_0002/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:45,181 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity975825011/.staging/job_local975825011_0002/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:45,282 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local975825011_0002/job_local975825011_0002.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:45,286 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local975825011_0002/job_local975825011_0002.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:45,287 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:45,288 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:45,288 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:45,288 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local975825011_0002
2015-02-10 22:48:45,290 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2015-02-10 22:48:45,293 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:45,293 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local975825011_0002_m_000000_0
2015-02-10 22:48:45,296 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:45,298 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:45,298 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@388eed76
2015-02-10 22:48:45,302 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:45,302 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:45,452 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:45,452 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:45,452 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:45,452 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:45,452 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:45,455 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:45,455 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:45,469 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:45,469 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'num']]
2015-02-10 22:48:45,470 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:45,472 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:45,472 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 88, total: 403, max: 672
2015-02-10 22:48:45,473 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 59% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:45,473 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:45,474 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 88, total: 403, max: 672
2015-02-10 22:48:45,474 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 59% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:45,476 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:45,477 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:45,477 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:45,477 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-10 22:48:45,477 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:48:45,482 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:45,485 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local975825011_0002_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:45,488 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:45,488 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local975825011_0002_m_000000_0' done.
2015-02-10 22:48:45,488 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local975825011_0002_m_000000_0
2015-02-10 22:48:45,489 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:45,489 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:45,490 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local975825011_0002_r_000000_0
2015-02-10 22:48:45,492 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:45,493 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@32552379
2015-02-10 22:48:45,495 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:45,496 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local975825011_0002_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:45,499 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#2 about to shuffle output of map attempt_local975825011_0002_m_000000_0 decomp: 87 len: 91 to MEMORY
2015-02-10 22:48:45,500 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local975825011_0002_m_000000_0
2015-02-10 22:48:45,500 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2015-02-10 22:48:45,501 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:45,503 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:45,503 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:45,507 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:45,508 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 22:48:45,510 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 87 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:45,511 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 91 bytes from disk
2015-02-10 22:48:45,511 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:45,511 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:45,512 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 22:48:45,514 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:45,515 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:45,515 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:45,529 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'num']]
2015-02-10 22:48:45,529 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring"]
2015-02-10 22:48:45,546 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local975825011_0002_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:45,548 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:45,548 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local975825011_0002_r_000000_0 is allowed to commit now
2015-02-10 22:48:45,549 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local975825011_0002_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring/_temporary/0/task_local975825011_0002_r_000000
2015-02-10 22:48:45,551 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:45,551 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local975825011_0002_r_000000_0' done.
2015-02-10 22:48:45,551 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local975825011_0002_r_000000_0
2015-02-10 22:48:45,551 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:45,581 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring/_temporary
2015-02-10 22:48:45,625 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:45,669 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:45,671 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/minbystring
2015-02-10 22:48:45,725 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:45,726 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:45,749 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:45,757 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:45,758 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:45,773 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs] starting
2015-02-10 22:48:45,774 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:48:45,774 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:45,774 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs]  sink: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage"]
2015-02-10 22:48:45,775 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs]  parallel execution is enabled: false
2015-02-10 22:48:45,775 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs]  starting jobs: 1
2015-02-10 22:48:45,775 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs]  allocating threads: 1
2015-02-10 22:48:45,776 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average-lhs+average-rhs] starting step: (1/1) ...persplatform/mergeaverage
2015-02-10 22:48:45,777 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:45,778 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:45,811 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:45,814 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:45,839 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 22:48:45,879 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local133005498_0003
2015-02-10 22:48:45,905 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity133005498/.staging/job_local133005498_0003/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:45,908 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity133005498/.staging/job_local133005498_0003/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:46,000 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local133005498_0003/job_local133005498_0003.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:46,003 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local133005498_0003/job_local133005498_0003.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:46,004 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:46,004 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average-lhs+average-rhs] submitted hadoop job: job_local133005498_0003
2015-02-10 22:48:46,005 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average-lhs+average-rhs] tracking url: http://localhost:8080/
2015-02-10 22:48:46,005 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:46,006 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:46,011 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:46,011 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local133005498_0003_m_000000_0
2015-02-10 22:48:46,015 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:46,018 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:46,018 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2eec0962
2015-02-10 22:48:46,021 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:46,021 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:46,170 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:46,171 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:46,171 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:46,171 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:46,171 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:46,174 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:46,174 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:46,191 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:46,191 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2015-02-10 22:48:46,192 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:46,194 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:46,194 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 122, total: 529, max: 672
2015-02-10 22:48:46,194 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 78% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:46,195 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:46,195 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 122, total: 529, max: 672
2015-02-10 22:48:46,195 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 78% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:46,196 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:46,196 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 122, total: 529, max: 672
2015-02-10 22:48:46,197 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 78% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:46,197 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:46,197 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 122, total: 529, max: 672
2015-02-10 22:48:46,197 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 78% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:46,199 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:46,200 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:46,200 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:46,200 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2015-02-10 22:48:46,200 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:46,205 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:46,208 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local133005498_0003_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:46,211 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:46,211 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local133005498_0003_m_000000_0' done.
2015-02-10 22:48:46,211 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local133005498_0003_m_000000_0
2015-02-10 22:48:46,211 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local133005498_0003_m_000001_0
2015-02-10 22:48:46,214 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:46,216 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt
2015-02-10 22:48:46,216 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@12072edc
2015-02-10 22:48:46,217 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:46,218 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:46,364 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:46,364 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:46,364 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:46,364 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:46,364 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:46,366 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:46,366 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:46,382 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:48:46,382 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2015-02-10 22:48:46,383 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:46,387 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:46,387 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 53, total: 561, max: 672
2015-02-10 22:48:46,389 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:46,389 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 53, total: 561, max: 672
2015-02-10 22:48:46,390 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:46,390 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 53, total: 561, max: 672
2015-02-10 22:48:46,390 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:46,391 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 53, total: 561, max: 672
2015-02-10 22:48:46,392 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:46,392 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:46,393 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:46,393 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2015-02-10 22:48:46,393 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:46,398 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:46,400 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local133005498_0003_m_000001_0 is done. And is in the process of committing
2015-02-10 22:48:46,403 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 22:48:46,403 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local133005498_0003_m_000001_0' done.
2015-02-10 22:48:46,403 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local133005498_0003_m_000001_0
2015-02-10 22:48:46,404 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:46,405 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:46,405 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local133005498_0003_r_000000_0
2015-02-10 22:48:46,408 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:46,408 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@54f690e4
2015-02-10 22:48:46,409 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:46,410 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local133005498_0003_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:46,414 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#3 about to shuffle output of map attempt_local133005498_0003_m_000000_0 decomp: 200 len: 204 to MEMORY
2015-02-10 22:48:46,414 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local133005498_0003_m_000000_0
2015-02-10 22:48:46,415 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->200
2015-02-10 22:48:46,418 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#3 about to shuffle output of map attempt_local133005498_0003_m_000001_0 decomp: 200 len: 204 to MEMORY
2015-02-10 22:48:46,418 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local133005498_0003_m_000001_0
2015-02-10 22:48:46,419 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 2, commitMemory -> 200, usedMemory ->400
2015-02-10 22:48:46,419 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:46,420 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:46,421 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:46,423 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 2 sorted segments
2015-02-10 22:48:46,424 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 2 segments left of total size: 382 bytes
2015-02-10 22:48:46,426 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 400 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:46,427 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 402 bytes from disk
2015-02-10 22:48:46,427 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:46,427 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:46,428 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 389 bytes
2015-02-10 22:48:46,428 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:46,429 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:46,429 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:46,444 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2015-02-10 22:48:46,444 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage"]
2015-02-10 22:48:46,460 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local133005498_0003_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:46,462 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:46,462 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local133005498_0003_r_000000_0 is allowed to commit now
2015-02-10 22:48:46,465 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local133005498_0003_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage/_temporary/0/task_local133005498_0003_r_000000
2015-02-10 22:48:46,467 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:46,467 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local133005498_0003_r_000000_0' done.
2015-02-10 22:48:46,467 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local133005498_0003_r_000000_0
2015-02-10 22:48:46,468 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:46,510 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage/_temporary
2015-02-10 22:48:46,549 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:46,588 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:46,590 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/mergeaverage
2015-02-10 22:48:46,624 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:46,625 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:46,639 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:46,643 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:46,644 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:46,659 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 22:48:46,659 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:46,659 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  sink: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall"]
2015-02-10 22:48:46,660 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  parallel execution is enabled: false
2015-02-10 22:48:46,660 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  starting jobs: 1
2015-02-10 22:48:46,660 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  allocating threads: 1
2015-02-10 22:48:46,661 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...helpersplatform/renameall
2015-02-10 22:48:46,662 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:46,663 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:46,699 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:46,724 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:46,749 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1647339523_0004
2015-02-10 22:48:46,774 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1647339523/.staging/job_local1647339523_0004/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:46,775 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1647339523/.staging/job_local1647339523_0004/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:46,850 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1647339523_0004/job_local1647339523_0004.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:46,852 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1647339523_0004/job_local1647339523_0004.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:46,853 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:46,853 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1647339523_0004
2015-02-10 22:48:46,854 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 22:48:46,854 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:46,855 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:46,860 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:46,860 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1647339523_0004_m_000000_0
2015-02-10 22:48:46,863 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:46,865 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt
2015-02-10 22:48:46,866 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@405a02ef
2015-02-10 22:48:46,867 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 22:48:46,878 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:46,878 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:46,887 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:46,887 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall"]
2015-02-10 22:48:46,891 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:46,891 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1647339523_0004_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:46,893 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:46,894 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1647339523_0004_m_000000_0 is allowed to commit now
2015-02-10 22:48:46,895 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1647339523_0004_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall/_temporary/0/task_local1647339523_0004_m_000000
2015-02-10 22:48:46,896 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt:0+19
2015-02-10 22:48:46,896 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1647339523_0004_m_000000_0' done.
2015-02-10 22:48:46,897 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1647339523_0004_m_000000_0
2015-02-10 22:48:46,897 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:47,243 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall/_temporary
2015-02-10 22:48:47,280 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:47,283 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/renameall
2015-02-10 22:48:47,318 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:47,319 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:47,344 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:47,348 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:47,369 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:47,381 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting
2015-02-10 22:48:47,382 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:47,382 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  sink: Hfs["TextDelimited[['count', 'count2']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount"]
2015-02-10 22:48:47,382 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  parallel execution is enabled: false
2015-02-10 22:48:47,382 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  starting jobs: 2
2015-02-10 22:48:47,383 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  allocating threads: 1
2015-02-10 22:48:47,383 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/2)
2015-02-10 22:48:47,385 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:47,386 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:47,419 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:47,442 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:47,476 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1304463010_0005
2015-02-10 22:48:47,497 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1304463010/.staging/job_local1304463010_0005/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:47,499 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1304463010/.staging/job_local1304463010_0005/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:47,559 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1304463010_0005/job_local1304463010_0005.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:47,561 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1304463010_0005/job_local1304463010_0005.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:47,561 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:47,562 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local1304463010_0005
2015-02-10 22:48:47,562 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-10 22:48:47,562 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:47,563 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:47,567 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:47,567 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1304463010_0005_m_000000_0
2015-02-10 22:48:47,569 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:47,571 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:47,571 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@54182d86
2015-02-10 22:48:47,573 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:47,573 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:47,650 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:47,650 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:47,650 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:47,650 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:47,650 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:47,652 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:47,652 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:47,666 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:47,667 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{1}:'char']]
2015-02-10 22:48:47,667 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:47,669 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:47,669 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 76, total: 219, max: 672
2015-02-10 22:48:47,669 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:47,670 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:47,670 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 76, total: 219, max: 672
2015-02-10 22:48:47,671 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:47,671 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:47,671 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 76, total: 219, max: 672
2015-02-10 22:48:47,672 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:47,672 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:47,672 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 76, total: 219, max: 672
2015-02-10 22:48:47,673 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:47,674 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:47,674 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:47,674 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:47,675 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 22:48:47,675 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:47,680 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:47,682 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1304463010_0005_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:47,686 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:47,687 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1304463010_0005_m_000000_0' done.
2015-02-10 22:48:47,687 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1304463010_0005_m_000000_0
2015-02-10 22:48:47,687 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:47,688 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:47,688 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1304463010_0005_r_000000_0
2015-02-10 22:48:47,690 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:47,690 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@25082661
2015-02-10 22:48:47,691 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:47,692 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1304463010_0005_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:47,694 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#4 about to shuffle output of map attempt_local1304463010_0005_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 22:48:47,695 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local1304463010_0005_m_000000_0
2015-02-10 22:48:47,695 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 22:48:47,696 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:47,697 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:47,697 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:47,702 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:47,702 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:47,704 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:47,704 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 22:48:47,705 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:47,705 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:47,705 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:47,706 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:47,706 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:47,706 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:47,719 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{1}:'char']]
2015-02-10 22:48:47,720 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['count', 'count2']]"][9561124881/count/]
2015-02-10 22:48:47,746 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:47,749 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:47,750 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 72, total: 219, max: 672
2015-02-10 22:48:47,750 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:47,751 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1304463010_0005_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:47,753 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:47,753 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1304463010_0005_r_000000_0 is allowed to commit now
2015-02-10 22:48:47,754 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1304463010_0005_r_000000_0' to file:/tmp/hadoop-teamcity/9561124881_count_9B0E1FAC21EB4717BBC6F58183F0801D/_temporary/0/task_local1304463010_0005_r_000000
2015-02-10 22:48:47,756 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:47,756 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1304463010_0005_r_000000_0' done.
2015-02-10 22:48:47,756 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1304463010_0005_r_000000_0
2015-02-10 22:48:47,757 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:47,786 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (2/2) ...elpersplatform/countcount
2015-02-10 22:48:47,786 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:47,788 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:47,832 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:47,865 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:47,879 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:47,879 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:47,887 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local167084815_0006
2015-02-10 22:48:47,912 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity167084815/.staging/job_local167084815_0006/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:47,913 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity167084815/.staging/job_local167084815_0006/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:47,969 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local167084815_0006/job_local167084815_0006.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:47,970 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local167084815_0006/job_local167084815_0006.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:47,971 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:47,972 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local167084815_0006
2015-02-10 22:48:47,972 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-10 22:48:47,972 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:47,973 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:47,976 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:47,977 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local167084815_0006_m_000000_0
2015-02-10 22:48:47,979 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:47,981 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/9561124881_count_9B0E1FAC21EB4717BBC6F58183F0801D/part-00000
2015-02-10 22:48:47,981 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@157ee3e5
2015-02-10 22:48:47,994 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:47,995 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:48,154 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:48,155 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:48,155 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:48,155 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:48,155 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:48,157 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:48,157 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:48,168 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['count', 'count2']]"][9561124881/count/]
2015-02-10 22:48:48,169 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{1}:'count']]
2015-02-10 22:48:48,171 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:48,171 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:48,172 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:48,172 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 21; bufvoid = 104857600
2015-02-10 22:48:48,172 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214388(104857552); length = 9/6553600
2015-02-10 22:48:48,175 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:48,177 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local167084815_0006_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:48,180 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/9561124881_count_9B0E1FAC21EB4717BBC6F58183F0801D/part-00000:0+112
2015-02-10 22:48:48,180 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local167084815_0006_m_000000_0' done.
2015-02-10 22:48:48,180 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local167084815_0006_m_000000_0
2015-02-10 22:48:48,181 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:48,181 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:48,181 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local167084815_0006_r_000000_0
2015-02-10 22:48:48,183 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:48,184 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@296068c9
2015-02-10 22:48:48,185 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:48,187 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local167084815_0006_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:48,189 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#5 about to shuffle output of map attempt_local167084815_0006_m_000000_0 decomp: 29 len: 33 to MEMORY
2015-02-10 22:48:48,189 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 29 bytes from map-output for attempt_local167084815_0006_m_000000_0
2015-02-10 22:48:48,190 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 29, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->29
2015-02-10 22:48:48,190 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:48,191 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:48,193 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:48,195 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:48,195 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 24 bytes
2015-02-10 22:48:48,197 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 29 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:48,197 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 33 bytes from disk
2015-02-10 22:48:48,197 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:48,198 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:48,198 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 24 bytes
2015-02-10 22:48:48,199 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:48,199 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:48,199 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:48,210 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{1}:'count']]
2015-02-10 22:48:48,210 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['count', 'count2']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount"]
2015-02-10 22:48:48,222 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local167084815_0006_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:48,227 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:48,227 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local167084815_0006_r_000000_0 is allowed to commit now
2015-02-10 22:48:48,229 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local167084815_0006_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount/_temporary/0/task_local167084815_0006_r_000000
2015-02-10 22:48:48,230 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:48,231 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local167084815_0006_r_000000_0' done.
2015-02-10 22:48:48,231 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local167084815_0006_r_000000_0
2015-02-10 22:48:48,232 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:48,256 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount/_temporary
2015-02-10 22:48:48,290 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:48,326 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:48,329 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/countcount
2015-02-10 22:48:48,362 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:48,362 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:48,382 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:48,388 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:48,388 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:48,400 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs] starting
2015-02-10 22:48:48,400 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:48:48,400 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:48,401 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs]  sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount"]
2015-02-10 22:48:48,401 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs]  parallel execution is enabled: false
2015-02-10 22:48:48,401 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs]  starting jobs: 1
2015-02-10 22:48:48,401 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs]  allocating threads: 1
2015-02-10 22:48:48,402 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs+count-rhs] starting step: (1/1) ...elpersplatform/mergecount
2015-02-10 22:48:48,403 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:48,404 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:48,433 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:48,435 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:48,456 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 22:48:48,475 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local882716914_0007
2015-02-10 22:48:48,503 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity882716914/.staging/job_local882716914_0007/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:48,505 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity882716914/.staging/job_local882716914_0007/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:48,560 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local882716914_0007/job_local882716914_0007.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:48,562 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local882716914_0007/job_local882716914_0007.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:48,563 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:48,563 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs+count-rhs] submitted hadoop job: job_local882716914_0007
2015-02-10 22:48:48,564 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:48,565 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:48,564 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs+count-rhs] tracking url: http://localhost:8080/
2015-02-10 22:48:48,569 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:48,569 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local882716914_0007_m_000000_0
2015-02-10 22:48:48,571 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:48,573 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:48,573 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@245f4ae
2015-02-10 22:48:48,575 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:48,575 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:48,725 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:48,725 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:48,725 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:48,726 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:48,726 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:48,727 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:48,727 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:48,741 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:48,741 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 22:48:48,742 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:48,744 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:48,744 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 114, total: 432, max: 672
2015-02-10 22:48:48,744 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 64% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:48,745 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:48,745 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 114, total: 432, max: 672
2015-02-10 22:48:48,746 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 64% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:48,746 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:48,746 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 114, total: 432, max: 672
2015-02-10 22:48:48,746 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 64% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:48,747 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:48,747 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 114, total: 432, max: 672
2015-02-10 22:48:48,747 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 64% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:48,749 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:48,749 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:48,749 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:48,749 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 22:48:48,749 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:48,754 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:48,756 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local882716914_0007_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:48,759 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:48,759 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local882716914_0007_m_000000_0' done.
2015-02-10 22:48:48,760 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local882716914_0007_m_000000_0
2015-02-10 22:48:48,760 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local882716914_0007_m_000001_0
2015-02-10 22:48:48,762 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:48,764 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt
2015-02-10 22:48:48,764 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@48ec77cb
2015-02-10 22:48:48,765 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:48,765 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:48,911 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:48,911 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:48,911 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:48,911 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:48,912 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:48,913 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:48,913 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:48,927 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:48:48,927 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 22:48:48,927 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:48,932 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:48,932 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 112, total: 532, max: 672
2015-02-10 22:48:48,932 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:48,933 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:48,933 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 112, total: 532, max: 672
2015-02-10 22:48:48,933 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:48,934 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:48,934 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 112, total: 532, max: 672
2015-02-10 22:48:48,934 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:48,935 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:48,935 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 112, total: 532, max: 672
2015-02-10 22:48:48,935 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:48,937 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:48,937 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:48,937 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:48,937 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 22:48:48,937 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:48,941 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:48,944 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local882716914_0007_m_000001_0 is done. And is in the process of committing
2015-02-10 22:48:48,946 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 22:48:48,946 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local882716914_0007_m_000001_0' done.
2015-02-10 22:48:48,946 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local882716914_0007_m_000001_0
2015-02-10 22:48:48,947 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:48,947 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:48,947 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local882716914_0007_r_000000_0
2015-02-10 22:48:48,950 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:48,950 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@c88a80e
2015-02-10 22:48:48,951 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:48,952 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local882716914_0007_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:48,956 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#6 about to shuffle output of map attempt_local882716914_0007_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 22:48:48,957 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local882716914_0007_m_000000_0
2015-02-10 22:48:48,957 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 22:48:48,960 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#6 about to shuffle output of map attempt_local882716914_0007_m_000001_0 decomp: 119 len: 123 to MEMORY
2015-02-10 22:48:48,960 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local882716914_0007_m_000001_0
2015-02-10 22:48:48,960 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 2, commitMemory -> 119, usedMemory ->238
2015-02-10 22:48:48,961 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:48,962 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:48,963 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:48,991 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 2 sorted segments
2015-02-10 22:48:48,991 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 2 segments left of total size: 220 bytes
2015-02-10 22:48:48,993 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 238 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:48,994 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 240 bytes from disk
2015-02-10 22:48:48,994 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:48,994 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:48,994 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 227 bytes
2015-02-10 22:48:48,995 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:48,995 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:48,996 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:49,009 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 22:48:49,009 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount"]
2015-02-10 22:48:49,022 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local882716914_0007_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:49,024 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:49,024 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local882716914_0007_r_000000_0 is allowed to commit now
2015-02-10 22:48:49,025 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local882716914_0007_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount/_temporary/0/task_local882716914_0007_r_000000
2015-02-10 22:48:49,026 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:49,026 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local882716914_0007_r_000000_0' done.
2015-02-10 22:48:49,027 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local882716914_0007_r_000000_0
2015-02-10 22:48:49,027 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:49,052 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount/_temporary
2015-02-10 22:48:49,087 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:49,121 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:49,123 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/mergecount
2015-02-10 22:48:49,151 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:49,151 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:49,163 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:49,166 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:49,166 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:49,176 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min] starting
2015-02-10 22:48:49,176 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:49,177 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  sink: Hfs["TextDelimited[['char', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbynullsafety"]
2015-02-10 22:48:49,177 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  parallel execution is enabled: false
2015-02-10 22:48:49,177 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  starting jobs: 1
2015-02-10 22:48:49,178 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  allocating threads: 1
2015-02-10 22:48:49,179 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] starting step: (1/1) ...splatform/minbynullsafety
2015-02-10 22:48:49,179 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:49,181 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:49,214 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:49,235 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:49,254 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local804643557_0008
2015-02-10 22:48:49,274 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity804643557/.staging/job_local804643557_0008/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:49,276 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity804643557/.staging/job_local804643557_0008/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:49,348 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local804643557_0008/job_local804643557_0008.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:49,350 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local804643557_0008/job_local804643557_0008.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:49,352 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:49,352 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] submitted hadoop job: job_local804643557_0008
2015-02-10 22:48:49,352 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] tracking url: http://localhost:8080/
2015-02-10 22:48:49,352 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:49,353 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:49,357 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:49,357 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local804643557_0008_m_000000_0
2015-02-10 22:48:49,360 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:49,362 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:49,362 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2cec849e
2015-02-10 22:48:49,364 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:49,365 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:49,696 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:49,697 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:49,698 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:49,698 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:49,698 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:49,700 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:49,700 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:49,712 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:49,712 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(min)[by:[{1}:'char']]
2015-02-10 22:48:49,713 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:49,715 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:49,716 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 152, total: 257, max: 672
2015-02-10 22:48:49,716 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 38% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:49,717 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:49,718 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 151, total: 257, max: 672
2015-02-10 22:48:49,718 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 38% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:49,719 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:49,719 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 151, total: 257, max: 672
2015-02-10 22:48:49,720 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 38% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:49,721 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:49,721 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 151, total: 257, max: 672
2015-02-10 22:48:49,721 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 38% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:49,724 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:49,725 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:49,725 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:49,726 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 22:48:49,726 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:49,730 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:49,733 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local804643557_0008_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:49,735 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:49,736 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local804643557_0008_m_000000_0' done.
2015-02-10 22:48:49,736 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local804643557_0008_m_000000_0
2015-02-10 22:48:49,737 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:49,737 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:49,738 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local804643557_0008_r_000000_0
2015-02-10 22:48:49,740 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:49,740 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7417d56b
2015-02-10 22:48:49,741 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:49,742 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local804643557_0008_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:49,744 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#7 about to shuffle output of map attempt_local804643557_0008_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 22:48:49,745 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local804643557_0008_m_000000_0
2015-02-10 22:48:49,745 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 22:48:49,746 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:49,748 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:49,748 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:49,751 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:49,751 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:49,753 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:49,754 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 22:48:49,754 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:49,754 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:49,755 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:49,756 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:49,757 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:49,757 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:49,768 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(min)[by:[{1}:'char']]
2015-02-10 22:48:49,768 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbynullsafety"]
2015-02-10 22:48:49,780 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local804643557_0008_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:49,782 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:49,783 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local804643557_0008_r_000000_0 is allowed to commit now
2015-02-10 22:48:49,784 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local804643557_0008_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbynullsafety/_temporary/0/task_local804643557_0008_r_000000
2015-02-10 22:48:49,785 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:49,786 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local804643557_0008_r_000000_0' done.
2015-02-10 22:48:49,786 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local804643557_0008_r_000000_0
2015-02-10 22:48:49,787 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:49,809 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbynullsafety/_temporary
2015-02-10 22:48:49,842 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:49,875 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:49,878 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/minbynullsafety
2015-02-10 22:48:49,903 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:49,903 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:49,913 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:49,915 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:49,916 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:49,926 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting
2015-02-10 22:48:49,927 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:49,927 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  sink: Hfs["TextDelimited[['num', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring"]
2015-02-10 22:48:49,927 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  parallel execution is enabled: false
2015-02-10 22:48:49,927 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  starting jobs: 1
2015-02-10 22:48:49,927 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  allocating threads: 1
2015-02-10 22:48:49,928 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...lpersplatform/maxbystring
2015-02-10 22:48:49,928 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:49,930 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:49,957 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:49,977 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:49,995 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local808692111_0009
2015-02-10 22:48:50,014 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity808692111/.staging/job_local808692111_0009/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:50,015 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity808692111/.staging/job_local808692111_0009/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:50,068 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local808692111_0009/job_local808692111_0009.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:50,069 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local808692111_0009/job_local808692111_0009.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:50,070 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:50,070 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local808692111_0009
2015-02-10 22:48:50,070 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:50,071 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2015-02-10 22:48:50,071 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:50,074 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:50,075 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local808692111_0009_m_000000_0
2015-02-10 22:48:50,076 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:50,078 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:50,078 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5ef4b65d
2015-02-10 22:48:50,080 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:50,080 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:50,230 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:50,231 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:50,231 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:50,231 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:50,231 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:50,232 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:50,233 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:50,243 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:50,243 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'num']]
2015-02-10 22:48:50,244 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:50,246 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:50,246 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 105, total: 357, max: 672
2015-02-10 22:48:50,246 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 53% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:50,247 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:50,247 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 105, total: 357, max: 672
2015-02-10 22:48:50,247 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 53% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:50,249 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:50,249 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:50,249 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:50,249 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-10 22:48:50,249 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:48:50,252 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:50,254 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local808692111_0009_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:50,257 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:50,257 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local808692111_0009_m_000000_0' done.
2015-02-10 22:48:50,257 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local808692111_0009_m_000000_0
2015-02-10 22:48:50,257 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:50,258 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:50,258 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local808692111_0009_r_000000_0
2015-02-10 22:48:50,260 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:50,260 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@669aa3f3
2015-02-10 22:48:50,261 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:50,262 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local808692111_0009_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:50,264 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#8 about to shuffle output of map attempt_local808692111_0009_m_000000_0 decomp: 87 len: 91 to MEMORY
2015-02-10 22:48:50,264 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local808692111_0009_m_000000_0
2015-02-10 22:48:50,264 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2015-02-10 22:48:50,265 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:50,266 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:50,266 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:50,268 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:50,269 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 22:48:50,270 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 87 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:50,271 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 91 bytes from disk
2015-02-10 22:48:50,271 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:50,271 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:50,271 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 22:48:50,272 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:50,272 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:50,273 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:50,283 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'num']]
2015-02-10 22:48:50,283 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring"]
2015-02-10 22:48:50,295 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local808692111_0009_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:50,297 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:50,297 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local808692111_0009_r_000000_0 is allowed to commit now
2015-02-10 22:48:50,298 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local808692111_0009_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring/_temporary/0/task_local808692111_0009_r_000000
2015-02-10 22:48:50,299 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:50,299 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local808692111_0009_r_000000_0' done.
2015-02-10 22:48:50,299 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local808692111_0009_r_000000_0
2015-02-10 22:48:50,299 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:50,323 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring/_temporary
2015-02-10 22:48:50,356 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:50,393 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:50,396 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/maxbystring
2015-02-10 22:48:50,433 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:50,434 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:50,446 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:50,449 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:50,451 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:50,467 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first] starting
2015-02-10 22:48:50,468 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  source: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 22:48:50,468 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  sink: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfields"]
2015-02-10 22:48:50,468 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  parallel execution is enabled: false
2015-02-10 22:48:50,468 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  starting jobs: 1
2015-02-10 22:48:50,468 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  allocating threads: 1
2015-02-10 22:48:50,469 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] starting step: (1/1) ...persplatform/firstnfields
2015-02-10 22:48:50,470 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:50,471 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:50,501 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:50,525 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:50,552 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local2092494425_0010
2015-02-10 22:48:50,578 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity2092494425/.staging/job_local2092494425_0010/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:50,580 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity2092494425/.staging/job_local2092494425_0010/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:50,650 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local2092494425_0010/job_local2092494425_0010.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:50,652 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local2092494425_0010/job_local2092494425_0010.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:50,653 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:50,653 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] submitted hadoop job: job_local2092494425_0010
2015-02-10 22:48:50,654 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:50,654 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] tracking url: http://localhost:8080/
2015-02-10 22:48:50,654 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:50,657 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:50,657 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local2092494425_0010_m_000000_0
2015-02-10 22:48:50,659 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:50,661 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 22:48:50,661 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@74a17378
2015-02-10 22:48:50,663 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:50,663 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:50,811 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:50,812 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:50,813 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:50,813 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:50,813 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:50,817 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:50,818 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:50,829 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 22:48:50,829 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(first)[by:[{1}:'num']]
2015-02-10 22:48:50,830 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:50,834 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:50,834 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 140, total: 457, max: 672
2015-02-10 22:48:50,835 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:50,836 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:50,837 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 140, total: 457, max: 672
2015-02-10 22:48:50,837 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:50,840 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:50,840 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:50,840 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:50,841 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 170; bufvoid = 104857600
2015-02-10 22:48:50,841 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:48:50,844 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:50,847 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local2092494425_0010_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:50,849 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 22:48:50,849 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2092494425_0010_m_000000_0' done.
2015-02-10 22:48:50,850 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local2092494425_0010_m_000000_0
2015-02-10 22:48:50,850 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:50,851 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:50,851 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local2092494425_0010_r_000000_0
2015-02-10 22:48:50,853 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:50,854 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@72ea9cf9
2015-02-10 22:48:50,854 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:50,855 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local2092494425_0010_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:50,858 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#9 about to shuffle output of map attempt_local2092494425_0010_m_000000_0 decomp: 182 len: 186 to MEMORY
2015-02-10 22:48:50,858 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 182 bytes from map-output for attempt_local2092494425_0010_m_000000_0
2015-02-10 22:48:50,858 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 182, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->182
2015-02-10 22:48:50,859 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:50,860 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:50,860 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:50,863 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:50,864 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 160 bytes
2015-02-10 22:48:50,866 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 182 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:50,866 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 186 bytes from disk
2015-02-10 22:48:50,867 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:50,867 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:50,867 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 160 bytes
2015-02-10 22:48:50,868 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:50,869 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:50,870 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:50,879 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(first)[by:[{1}:'num']]
2015-02-10 22:48:50,880 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfields"]
2015-02-10 22:48:50,891 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local2092494425_0010_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:50,893 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:50,893 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local2092494425_0010_r_000000_0 is allowed to commit now
2015-02-10 22:48:50,894 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local2092494425_0010_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfields/_temporary/0/task_local2092494425_0010_r_000000
2015-02-10 22:48:50,896 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:50,896 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2092494425_0010_r_000000_0' done.
2015-02-10 22:48:50,896 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local2092494425_0010_r_000000_0
2015-02-10 22:48:50,897 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:50,916 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfields/_temporary
2015-02-10 22:48:50,947 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:50,949 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/firstnfields
2015-02-10 22:48:50,975 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:50,976 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:50,984 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:50,986 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:50,987 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:50,996 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] starting
2015-02-10 22:48:50,996 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:50,996 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  sink: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields"]
2015-02-10 22:48:50,997 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  parallel execution is enabled: false
2015-02-10 22:48:50,997 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  starting jobs: 1
2015-02-10 22:48:50,997 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  allocating threads: 1
2015-02-10 22:48:50,999 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] starting step: (1/1) ...persplatform/coercefields
2015-02-10 22:48:50,999 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:51,001 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:51,025 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:51,045 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:51,062 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1359104465_0011
2015-02-10 22:48:51,083 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1359104465/.staging/job_local1359104465_0011/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:51,084 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1359104465/.staging/job_local1359104465_0011/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:51,138 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1359104465_0011/job_local1359104465_0011.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:51,139 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1359104465_0011/job_local1359104465_0011.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:51,140 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:51,140 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] submitted hadoop job: job_local1359104465_0011
2015-02-10 22:48:51,141 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:51,141 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] tracking url: http://localhost:8080/
2015-02-10 22:48:51,141 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:51,144 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:51,144 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1359104465_0011_m_000000_0
2015-02-10 22:48:51,146 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:51,148 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt
2015-02-10 22:48:51,148 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@41c17a8c
2015-02-10 22:48:51,150 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 22:48:51,158 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:51,159 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:51,169 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:51,169 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields"]
2015-02-10 22:48:51,181 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:51,182 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1359104465_0011_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:51,183 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:51,183 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1359104465_0011_m_000000_0 is allowed to commit now
2015-02-10 22:48:51,184 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1359104465_0011_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields/_temporary/0/task_local1359104465_0011_m_000000
2015-02-10 22:48:51,186 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt:0+19
2015-02-10 22:48:51,186 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1359104465_0011_m_000000_0' done.
2015-02-10 22:48:51,186 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1359104465_0011_m_000000_0
2015-02-10 22:48:51,186 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:51,206 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields/_temporary
2015-02-10 22:48:51,241 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:51,243 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/coercefields
2015-02-10 22:48:51,266 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:51,267 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:51,278 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:51,281 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:51,282 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:51,289 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting
2015-02-10 22:48:51,289 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:51,290 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count"]
2015-02-10 22:48:51,290 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  parallel execution is enabled: false
2015-02-10 22:48:51,290 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  starting jobs: 1
2015-02-10 22:48:51,290 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  allocating threads: 1
2015-02-10 22:48:51,291 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...mblyhelpersplatform/count
2015-02-10 22:48:51,292 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:51,294 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:51,321 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:51,342 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:51,359 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1277683109_0012
2015-02-10 22:48:51,386 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1277683109/.staging/job_local1277683109_0012/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:51,387 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1277683109/.staging/job_local1277683109_0012/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:51,440 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1277683109_0012/job_local1277683109_0012.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:51,441 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1277683109_0012/job_local1277683109_0012.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:51,442 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:51,442 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local1277683109_0012
2015-02-10 22:48:51,442 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:51,442 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-10 22:48:51,443 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:51,445 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:51,445 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1277683109_0012_m_000000_0
2015-02-10 22:48:51,447 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:51,449 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:51,449 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3358fd70
2015-02-10 22:48:51,450 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:51,451 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:51,516 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:51,516 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:51,516 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:51,516 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:51,516 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:51,518 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:51,518 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:51,528 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:51,528 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{1}:'char']]
2015-02-10 22:48:51,529 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:51,531 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:51,531 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 77, total: 497, max: 672
2015-02-10 22:48:51,531 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 73% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:51,532 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:51,532 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 77, total: 497, max: 672
2015-02-10 22:48:51,533 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 73% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:51,534 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:51,534 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 77, total: 497, max: 672
2015-02-10 22:48:51,534 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 73% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:51,535 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:51,536 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 77, total: 497, max: 672
2015-02-10 22:48:51,536 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 73% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:51,538 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:51,538 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:51,538 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:51,539 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 22:48:51,539 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:51,543 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:51,546 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1277683109_0012_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:51,548 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:51,549 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1277683109_0012_m_000000_0' done.
2015-02-10 22:48:51,549 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1277683109_0012_m_000000_0
2015-02-10 22:48:51,550 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:51,550 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:51,551 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1277683109_0012_r_000000_0
2015-02-10 22:48:51,553 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:51,554 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@402fbd59
2015-02-10 22:48:51,555 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:51,557 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1277683109_0012_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:51,559 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#10 about to shuffle output of map attempt_local1277683109_0012_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 22:48:51,560 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local1277683109_0012_m_000000_0
2015-02-10 22:48:51,560 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 22:48:51,560 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:51,562 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:51,563 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:51,566 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:51,566 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:51,568 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:51,569 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 22:48:51,569 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:51,570 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:51,570 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:51,571 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:51,572 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:51,572 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:51,583 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{1}:'char']]
2015-02-10 22:48:51,584 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count"]
2015-02-10 22:48:51,597 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1277683109_0012_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:51,600 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:51,600 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1277683109_0012_r_000000_0 is allowed to commit now
2015-02-10 22:48:51,601 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1277683109_0012_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count/_temporary/0/task_local1277683109_0012_r_000000
2015-02-10 22:48:51,602 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:51,603 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1277683109_0012_r_000000_0' done.
2015-02-10 22:48:51,603 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1277683109_0012_r_000000_0
2015-02-10 22:48:51,603 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:51,626 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count/_temporary
2015-02-10 22:48:51,672 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:51,704 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:51,706 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/count
2015-02-10 22:48:51,728 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:51,729 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:51,737 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:51,739 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:51,740 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:51,745 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting
2015-02-10 22:48:51,746 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:51,746 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  sink: Hfs["TextDelimited[['char', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby"]
2015-02-10 22:48:51,746 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  parallel execution is enabled: false
2015-02-10 22:48:51,747 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  starting jobs: 1
2015-02-10 22:48:51,747 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  allocating threads: 1
2015-02-10 22:48:51,748 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...mblyhelpersplatform/maxby
2015-02-10 22:48:51,749 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:51,750 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:51,775 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:51,796 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:51,812 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local971840396_0013
2015-02-10 22:48:51,830 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity971840396/.staging/job_local971840396_0013/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:51,831 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity971840396/.staging/job_local971840396_0013/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:51,881 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local971840396_0013/job_local971840396_0013.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:51,883 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local971840396_0013/job_local971840396_0013.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:51,883 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:51,884 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local971840396_0013
2015-02-10 22:48:51,884 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:51,884 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2015-02-10 22:48:51,884 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:51,887 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:51,887 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local971840396_0013_m_000000_0
2015-02-10 22:48:51,890 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:51,892 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:51,892 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3ec1e10c
2015-02-10 22:48:51,894 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:51,894 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:52,040 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:52,041 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:52,041 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:52,041 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:52,041 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:52,043 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:52,043 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:52,052 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:52,052 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'char']]
2015-02-10 22:48:52,053 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:52,054 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:52,054 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 144, total: 599, max: 672
2015-02-10 22:48:52,055 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:52,055 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 144, total: 599, max: 672
2015-02-10 22:48:52,056 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:52,056 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 144, total: 599, max: 672
2015-02-10 22:48:52,057 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:52,057 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 144, total: 599, max: 672
2015-02-10 22:48:52,058 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:52,059 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:52,059 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:52,059 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 135; bufvoid = 104857600
2015-02-10 22:48:52,059 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:52,062 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:52,064 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local971840396_0013_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:52,066 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:52,067 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local971840396_0013_m_000000_0' done.
2015-02-10 22:48:52,067 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local971840396_0013_m_000000_0
2015-02-10 22:48:52,067 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:52,068 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:52,068 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local971840396_0013_r_000000_0
2015-02-10 22:48:52,069 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:52,070 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@1a3ae8ac
2015-02-10 22:48:52,070 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:52,071 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local971840396_0013_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:52,074 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#11 about to shuffle output of map attempt_local971840396_0013_m_000000_0 decomp: 155 len: 159 to MEMORY
2015-02-10 22:48:52,074 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 155 bytes from map-output for attempt_local971840396_0013_m_000000_0
2015-02-10 22:48:52,074 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 155, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->155
2015-02-10 22:48:52,075 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:52,076 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:52,077 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:52,079 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:52,079 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2015-02-10 22:48:52,080 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 155 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:52,081 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 159 bytes from disk
2015-02-10 22:48:52,081 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:52,081 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:52,081 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2015-02-10 22:48:52,082 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:52,082 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:52,083 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:52,091 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'char']]
2015-02-10 22:48:52,091 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby"]
2015-02-10 22:48:52,102 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local971840396_0013_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:52,104 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:52,104 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local971840396_0013_r_000000_0 is allowed to commit now
2015-02-10 22:48:52,106 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local971840396_0013_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby/_temporary/0/task_local971840396_0013_r_000000
2015-02-10 22:48:52,108 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:52,108 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local971840396_0013_r_000000_0' done.
2015-02-10 22:48:52,108 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local971840396_0013_r_000000_0
2015-02-10 22:48:52,108 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:52,126 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby/_temporary
2015-02-10 22:48:52,159 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:52,194 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:52,196 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/maxby
2015-02-10 22:48:52,218 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:52,218 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:52,227 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:52,229 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:52,230 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:52,235 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min] starting
2015-02-10 22:48:52,235 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:52,235 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  sink: Hfs["TextDelimited[['char', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby"]
2015-02-10 22:48:52,236 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  parallel execution is enabled: false
2015-02-10 22:48:52,236 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  starting jobs: 1
2015-02-10 22:48:52,236 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  allocating threads: 1
2015-02-10 22:48:52,236 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] starting step: (1/1) ...mblyhelpersplatform/minby
2015-02-10 22:48:52,237 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:52,238 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:52,264 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:52,282 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:52,298 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local863072059_0014
2015-02-10 22:48:52,318 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity863072059/.staging/job_local863072059_0014/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:52,319 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity863072059/.staging/job_local863072059_0014/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:52,370 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local863072059_0014/job_local863072059_0014.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:52,372 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local863072059_0014/job_local863072059_0014.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:52,372 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:52,373 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] submitted hadoop job: job_local863072059_0014
2015-02-10 22:48:52,373 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:52,373 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] tracking url: http://localhost:8080/
2015-02-10 22:48:52,373 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:52,376 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:52,376 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local863072059_0014_m_000000_0
2015-02-10 22:48:52,378 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:52,380 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:52,380 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@14d6c410
2015-02-10 22:48:52,381 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:52,381 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:52,411 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:52,411 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:52,413 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:52,413 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:52,413 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:52,414 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:52,414 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:52,423 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:52,424 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(min)[by:[{1}:'char']]
2015-02-10 22:48:52,424 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:52,426 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:52,426 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 181, total: 689, max: 689
2015-02-10 22:48:52,427 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:52,427 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 181, total: 689, max: 689
2015-02-10 22:48:52,427 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:52,428 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 181, total: 689, max: 689
2015-02-10 22:48:52,428 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:52,428 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 181, total: 689, max: 689
2015-02-10 22:48:52,430 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:52,430 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:52,430 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:52,430 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 135; bufvoid = 104857600
2015-02-10 22:48:52,430 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:52,434 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:52,436 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local863072059_0014_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:52,441 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:52,441 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local863072059_0014_m_000000_0' done.
2015-02-10 22:48:52,441 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local863072059_0014_m_000000_0
2015-02-10 22:48:52,442 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:52,442 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:52,443 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local863072059_0014_r_000000_0
2015-02-10 22:48:52,444 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:52,444 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5c4737a3
2015-02-10 22:48:52,445 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=505957568, maxSingleShuffleLimit=126489392, mergeThreshold=333932000, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:52,446 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local863072059_0014_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:52,448 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#12 about to shuffle output of map attempt_local863072059_0014_m_000000_0 decomp: 155 len: 159 to MEMORY
2015-02-10 22:48:52,449 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 155 bytes from map-output for attempt_local863072059_0014_m_000000_0
2015-02-10 22:48:52,449 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 155, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->155
2015-02-10 22:48:52,450 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:52,451 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:52,452 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:52,454 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:52,454 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2015-02-10 22:48:52,455 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 155 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:52,456 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 159 bytes from disk
2015-02-10 22:48:52,456 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:52,456 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:52,456 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2015-02-10 22:48:52,457 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:52,457 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:52,457 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:52,466 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(min)[by:[{1}:'char']]
2015-02-10 22:48:52,466 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby"]
2015-02-10 22:48:52,476 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local863072059_0014_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:52,478 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:52,478 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local863072059_0014_r_000000_0 is allowed to commit now
2015-02-10 22:48:52,479 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local863072059_0014_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby/_temporary/0/task_local863072059_0014_r_000000
2015-02-10 22:48:52,480 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:52,480 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local863072059_0014_r_000000_0' done.
2015-02-10 22:48:52,480 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local863072059_0014_r_000000_0
2015-02-10 22:48:52,481 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:52,501 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby/_temporary
2015-02-10 22:48:52,536 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:52,569 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:52,570 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/minby
2015-02-10 22:48:52,597 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:52,598 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:52,609 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:52,612 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:52,613 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:52,620 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] starting
2015-02-10 22:48:52,621 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:52,621 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  sink: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum"]
2015-02-10 22:48:52,621 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  parallel execution is enabled: false
2015-02-10 22:48:52,621 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  starting jobs: 1
2015-02-10 22:48:52,621 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  allocating threads: 1
2015-02-10 22:48:52,622 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] starting step: (1/1) ...semblyhelpersplatform/sum
2015-02-10 22:48:52,623 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:52,624 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:52,649 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:52,674 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:52,697 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local965179374_0015
2015-02-10 22:48:52,722 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity965179374/.staging/job_local965179374_0015/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:52,725 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity965179374/.staging/job_local965179374_0015/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:52,794 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local965179374_0015/job_local965179374_0015.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:52,797 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local965179374_0015/job_local965179374_0015.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:52,798 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:52,798 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:52,799 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:52,798 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] submitted hadoop job: job_local965179374_0015
2015-02-10 22:48:52,799 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] tracking url: http://localhost:8080/
2015-02-10 22:48:52,802 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:52,802 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local965179374_0015_m_000000_0
2015-02-10 22:48:52,804 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:52,807 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:52,807 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7d786789
2015-02-10 22:48:52,809 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:52,810 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:52,840 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:52,840 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:52,840 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:52,840 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:52,841 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:52,842 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:52,842 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:52,852 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:52,852 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum)[by:[{1}:'char']]
2015-02-10 22:48:52,853 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:52,854 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:52,854 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 184, total: 692, max: 692
2015-02-10 22:48:52,855 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:52,856 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 184, total: 692, max: 692
2015-02-10 22:48:52,856 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:52,856 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 184, total: 692, max: 692
2015-02-10 22:48:52,857 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:52,857 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 184, total: 692, max: 692
2015-02-10 22:48:52,858 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:52,859 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:52,859 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:52,859 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 22:48:52,859 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:52,862 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:52,864 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local965179374_0015_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:52,870 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:52,870 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local965179374_0015_m_000000_0' done.
2015-02-10 22:48:52,870 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local965179374_0015_m_000000_0
2015-02-10 22:48:52,871 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:52,871 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:52,871 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local965179374_0015_r_000000_0
2015-02-10 22:48:52,873 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:52,874 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@65a64b69
2015-02-10 22:48:52,874 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=508434848, maxSingleShuffleLimit=127108712, mergeThreshold=335567008, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:52,876 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local965179374_0015_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:52,878 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#13 about to shuffle output of map attempt_local965179374_0015_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 22:48:52,879 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local965179374_0015_m_000000_0
2015-02-10 22:48:52,879 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 22:48:52,880 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:52,881 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:52,881 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:52,885 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:52,885 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:52,887 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:52,887 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 22:48:52,887 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:52,888 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:52,888 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:52,889 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:52,889 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:52,889 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:52,898 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(sum)[by:[{1}:'char']]
2015-02-10 22:48:52,899 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum"]
2015-02-10 22:48:52,910 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local965179374_0015_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:52,912 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:52,912 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local965179374_0015_r_000000_0 is allowed to commit now
2015-02-10 22:48:52,913 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local965179374_0015_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum/_temporary/0/task_local965179374_0015_r_000000
2015-02-10 22:48:52,914 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:52,914 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local965179374_0015_r_000000_0' done.
2015-02-10 22:48:52,914 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local965179374_0015_r_000000_0
2015-02-10 22:48:52,915 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:52,940 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum/_temporary
2015-02-10 22:48:52,971 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:53,005 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:53,007 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/sum
2015-02-10 22:48:53,031 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:53,032 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:53,044 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:53,048 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:53,048 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:53,054 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs] starting
2015-02-10 22:48:53,054 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:48:53,054 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:53,054 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs]  sink: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum"]
2015-02-10 22:48:53,054 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs]  parallel execution is enabled: false
2015-02-10 22:48:53,055 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs]  starting jobs: 1
2015-02-10 22:48:53,055 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs]  allocating threads: 1
2015-02-10 22:48:53,055 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum-lhs+sum-rhs] starting step: (1/1) ...yhelpersplatform/mergesum
2015-02-10 22:48:53,056 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:53,057 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:53,082 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:53,083 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:53,102 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 22:48:53,124 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1315961791_0016
2015-02-10 22:48:53,146 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1315961791/.staging/job_local1315961791_0016/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:53,147 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1315961791/.staging/job_local1315961791_0016/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:53,200 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1315961791_0016/job_local1315961791_0016.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:53,201 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1315961791_0016/job_local1315961791_0016.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:53,202 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:53,202 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:53,203 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum-lhs+sum-rhs] submitted hadoop job: job_local1315961791_0016
2015-02-10 22:48:53,203 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum-lhs+sum-rhs] tracking url: http://localhost:8080/
2015-02-10 22:48:53,203 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:53,205 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:53,206 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1315961791_0016_m_000000_0
2015-02-10 22:48:53,207 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:53,208 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:53,209 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@73b829e7
2015-02-10 22:48:53,210 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:53,210 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:53,238 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:53,238 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:53,238 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:53,238 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:53,238 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:53,240 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:53,240 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:53,251 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:53,251 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2015-02-10 22:48:53,252 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:53,253 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:53,254 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 231, total: 739, max: 739
2015-02-10 22:48:53,255 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:53,255 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 231, total: 739, max: 739
2015-02-10 22:48:53,255 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:53,256 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 231, total: 739, max: 739
2015-02-10 22:48:53,256 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:53,256 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 230, total: 739, max: 739
2015-02-10 22:48:53,258 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:53,258 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:53,258 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:53,258 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 22:48:53,258 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:53,261 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:53,263 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1315961791_0016_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:53,265 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:53,266 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1315961791_0016_m_000000_0' done.
2015-02-10 22:48:53,266 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1315961791_0016_m_000000_0
2015-02-10 22:48:53,266 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1315961791_0016_m_000001_0
2015-02-10 22:48:53,267 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:53,269 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt
2015-02-10 22:48:53,269 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6cab6dcb
2015-02-10 22:48:53,270 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:53,270 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:53,351 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:53,352 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:53,352 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:53,352 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:53,352 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:53,353 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:53,353 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:53,365 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:48:53,365 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2015-02-10 22:48:53,366 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:53,370 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:53,370 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 129, total: 739, max: 739
2015-02-10 22:48:53,372 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:53,372 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 129, total: 739, max: 739
2015-02-10 22:48:53,372 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:53,373 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 129, total: 739, max: 739
2015-02-10 22:48:53,373 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:53,373 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 129, total: 739, max: 739
2015-02-10 22:48:53,375 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:53,375 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:53,375 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:53,375 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 22:48:53,375 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:53,379 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:53,381 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1315961791_0016_m_000001_0 is done. And is in the process of committing
2015-02-10 22:48:53,383 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 22:48:53,383 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1315961791_0016_m_000001_0' done.
2015-02-10 22:48:53,384 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1315961791_0016_m_000001_0
2015-02-10 22:48:53,384 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:53,384 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:53,385 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1315961791_0016_r_000000_0
2015-02-10 22:48:53,386 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:53,387 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7faea002
2015-02-10 22:48:53,389 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=542932992, maxSingleShuffleLimit=135733248, mergeThreshold=358335776, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:53,390 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1315961791_0016_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:53,392 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#14 about to shuffle output of map attempt_local1315961791_0016_m_000001_0 decomp: 119 len: 123 to MEMORY
2015-02-10 22:48:53,393 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local1315961791_0016_m_000001_0
2015-02-10 22:48:53,393 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 22:48:53,396 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#14 about to shuffle output of map attempt_local1315961791_0016_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 22:48:53,396 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local1315961791_0016_m_000000_0
2015-02-10 22:48:53,397 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 2, commitMemory -> 119, usedMemory ->238
2015-02-10 22:48:53,397 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:53,398 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:53,398 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:53,401 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 2 sorted segments
2015-02-10 22:48:53,401 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 2 segments left of total size: 220 bytes
2015-02-10 22:48:53,403 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 238 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:53,403 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 240 bytes from disk
2015-02-10 22:48:53,403 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:53,404 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:53,404 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 227 bytes
2015-02-10 22:48:53,405 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:53,405 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:53,405 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:53,416 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2015-02-10 22:48:53,417 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum"]
2015-02-10 22:48:53,428 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1315961791_0016_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:53,430 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:53,430 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1315961791_0016_r_000000_0 is allowed to commit now
2015-02-10 22:48:53,431 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1315961791_0016_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum/_temporary/0/task_local1315961791_0016_r_000000
2015-02-10 22:48:53,432 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:53,432 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1315961791_0016_r_000000_0' done.
2015-02-10 22:48:53,432 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1315961791_0016_r_000000_0
2015-02-10 22:48:53,433 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:53,453 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum/_temporary
2015-02-10 22:48:53,487 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:53,525 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:53,527 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/mergesum
2015-02-10 22:48:53,553 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:53,554 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:53,562 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:53,574 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:53,574 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:53,580 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 22:48:53,580 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:53,580 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  sink: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow"]
2015-02-10 22:48:53,580 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  parallel execution is enabled: false
2015-02-10 22:48:53,580 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  starting jobs: 1
2015-02-10 22:48:53,581 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  allocating threads: 1
2015-02-10 22:48:53,581 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...ersplatform/discardnarrow
2015-02-10 22:48:53,582 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:53,583 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:53,606 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:53,625 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:53,641 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local60824353_0017
2015-02-10 22:48:53,660 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity60824353/.staging/job_local60824353_0017/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:53,661 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity60824353/.staging/job_local60824353_0017/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:53,707 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local60824353_0017/job_local60824353_0017.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:53,708 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local60824353_0017/job_local60824353_0017.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:53,709 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:53,709 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local60824353_0017
2015-02-10 22:48:53,709 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:53,710 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 22:48:53,710 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:53,712 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:53,713 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local60824353_0017_m_000000_0
2015-02-10 22:48:53,715 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:53,716 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt
2015-02-10 22:48:53,717 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@463b7f4e
2015-02-10 22:48:53,718 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 22:48:53,726 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:53,726 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:53,732 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:53,732 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow"]
2015-02-10 22:48:53,736 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:53,736 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local60824353_0017_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:53,737 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:53,737 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local60824353_0017_m_000000_0 is allowed to commit now
2015-02-10 22:48:53,738 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local60824353_0017_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow/_temporary/0/task_local60824353_0017_m_000000
2015-02-10 22:48:53,739 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt:0+19
2015-02-10 22:48:53,739 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local60824353_0017_m_000000_0' done.
2015-02-10 22:48:53,740 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local60824353_0017_m_000000_0
2015-02-10 22:48:53,740 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:53,754 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow/_temporary
2015-02-10 22:48:53,785 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:53,787 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/discardnarrow
2015-02-10 22:48:53,807 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:53,807 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:53,814 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:53,816 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:53,817 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:53,820 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 22:48:53,821 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:53,821 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  sink: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename"]
2015-02-10 22:48:53,821 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  parallel execution is enabled: false
2015-02-10 22:48:53,821 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  starting jobs: 1
2015-02-10 22:48:53,821 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  allocating threads: 1
2015-02-10 22:48:53,822 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...blyhelpersplatform/rename
2015-02-10 22:48:53,823 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:53,824 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:53,847 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:53,866 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:53,882 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local541929438_0018
2015-02-10 22:48:53,900 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity541929438/.staging/job_local541929438_0018/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:53,902 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity541929438/.staging/job_local541929438_0018/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:53,949 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local541929438_0018/job_local541929438_0018.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:53,950 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local541929438_0018/job_local541929438_0018.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:53,951 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:53,951 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local541929438_0018
2015-02-10 22:48:53,951 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:53,951 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 22:48:53,952 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:53,954 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:53,954 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local541929438_0018_m_000000_0
2015-02-10 22:48:53,956 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:53,957 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt
2015-02-10 22:48:53,958 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@34ac3c11
2015-02-10 22:48:53,959 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 22:48:53,966 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:53,966 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:53,972 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:53,972 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename"]
2015-02-10 22:48:53,975 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:54,027 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local541929438_0018_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:54,029 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:54,029 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local541929438_0018_m_000000_0 is allowed to commit now
2015-02-10 22:48:54,030 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local541929438_0018_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename/_temporary/0/task_local541929438_0018_m_000000
2015-02-10 22:48:54,031 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt:0+19
2015-02-10 22:48:54,031 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local541929438_0018_m_000000_0' done.
2015-02-10 22:48:54,031 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local541929438_0018_m_000000_0
2015-02-10 22:48:54,031 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:54,046 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename/_temporary
2015-02-10 22:48:54,078 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:54,080 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/rename
2015-02-10 22:48:54,105 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:54,106 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:54,121 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:54,124 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:54,125 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:54,132 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name] starting
2015-02-10 22:48:54,133 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:48:54,133 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:54,133 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true"]
2015-02-10 22:48:54,133 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  parallel execution is enabled: false
2015-02-10 22:48:54,133 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  starting jobs: 1
2015-02-10 22:48:54,133 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  allocating threads: 1
2015-02-10 22:48:54,134 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] starting step: (1/1) ...splatform/multimerge+true
2015-02-10 22:48:54,135 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:54,136 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:54,160 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:54,161 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:54,181 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 22:48:54,197 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1534238509_0019
2015-02-10 22:48:54,215 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1534238509/.staging/job_local1534238509_0019/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:54,217 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1534238509/.staging/job_local1534238509_0019/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:54,265 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1534238509_0019/job_local1534238509_0019.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:54,267 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1534238509_0019/job_local1534238509_0019.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:54,268 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:54,269 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:54,269 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:54,269 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] submitted hadoop job: job_local1534238509_0019
2015-02-10 22:48:54,270 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] tracking url: http://localhost:8080/
2015-02-10 22:48:54,272 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:54,272 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1534238509_0019_m_000000_0
2015-02-10 22:48:54,273 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:54,274 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:54,275 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2b76fbc2
2015-02-10 22:48:54,276 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:54,276 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:54,303 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:54,303 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:54,303 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:54,303 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:54,304 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:54,306 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:54,306 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:54,319 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:54,319 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2015-02-10 22:48:54,319 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:54,321 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:54,321 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 750, max: 750
2015-02-10 22:48:54,322 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:54,322 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 750, max: 750
2015-02-10 22:48:54,323 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:54,323 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 750, max: 750
2015-02-10 22:48:54,324 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:54,324 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 750, max: 750
2015-02-10 22:48:54,326 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:54,326 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:54,326 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:54,326 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2015-02-10 22:48:54,326 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:54,329 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:54,331 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1534238509_0019_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:54,333 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:54,334 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1534238509_0019_m_000000_0' done.
2015-02-10 22:48:54,334 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1534238509_0019_m_000000_0
2015-02-10 22:48:54,334 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1534238509_0019_m_000001_0
2015-02-10 22:48:54,336 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:54,337 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt
2015-02-10 22:48:54,338 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3aa8c31b
2015-02-10 22:48:54,339 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:54,339 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:54,360 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:54,361 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:54,361 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:54,361 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:54,362 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:54,364 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:54,364 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:54,376 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:48:54,377 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2015-02-10 22:48:54,377 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:54,382 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:54,382 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 136, total: 750, max: 750
2015-02-10 22:48:54,384 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:54,384 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 136, total: 750, max: 750
2015-02-10 22:48:54,385 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:54,385 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 136, total: 750, max: 750
2015-02-10 22:48:54,386 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:54,386 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 136, total: 750, max: 750
2015-02-10 22:48:54,388 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:54,388 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:54,388 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:54,389 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2015-02-10 22:48:54,389 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:54,392 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:54,395 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1534238509_0019_m_000001_0 is done. And is in the process of committing
2015-02-10 22:48:54,397 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 22:48:54,397 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1534238509_0019_m_000001_0' done.
2015-02-10 22:48:54,398 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1534238509_0019_m_000001_0
2015-02-10 22:48:54,398 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:54,399 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:54,399 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1534238509_0019_r_000000_0
2015-02-10 22:48:54,401 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:54,402 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@3b5b0734
2015-02-10 22:48:54,402 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=550869376, maxSingleShuffleLimit=137717344, mergeThreshold=363573792, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:54,403 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1534238509_0019_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:54,407 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#15 about to shuffle output of map attempt_local1534238509_0019_m_000001_0 decomp: 236 len: 240 to MEMORY
2015-02-10 22:48:54,408 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local1534238509_0019_m_000001_0
2015-02-10 22:48:54,408 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->236
2015-02-10 22:48:54,411 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#15 about to shuffle output of map attempt_local1534238509_0019_m_000000_0 decomp: 236 len: 240 to MEMORY
2015-02-10 22:48:54,411 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local1534238509_0019_m_000000_0
2015-02-10 22:48:54,411 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 2, commitMemory -> 236, usedMemory ->472
2015-02-10 22:48:54,412 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:54,413 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:54,413 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:54,416 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 2 sorted segments
2015-02-10 22:48:54,417 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 2 segments left of total size: 454 bytes
2015-02-10 22:48:54,419 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 472 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:54,420 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 474 bytes from disk
2015-02-10 22:48:54,420 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:54,420 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:54,421 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 461 bytes
2015-02-10 22:48:54,422 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:54,423 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:54,423 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:54,436 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(name)[by:[{1}:'char']]
2015-02-10 22:48:54,436 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true"]
2015-02-10 22:48:54,450 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1534238509_0019_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:54,452 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:54,452 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1534238509_0019_r_000000_0 is allowed to commit now
2015-02-10 22:48:54,453 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1534238509_0019_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true/_temporary/0/task_local1534238509_0019_r_000000
2015-02-10 22:48:54,455 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:54,455 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1534238509_0019_r_000000_0' done.
2015-02-10 22:48:54,456 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1534238509_0019_r_000000_0
2015-02-10 22:48:54,456 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:54,477 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true/_temporary
2015-02-10 22:48:54,511 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:54,549 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:54,551 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/multimerge+true
2015-02-10 22:48:54,579 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:54,580 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:54,607 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:54,609 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:54,616 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:54,622 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [second] starting
2015-02-10 22:48:54,623 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [second]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:54,623 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [second]  sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount"]
2015-02-10 22:48:54,623 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [second]  parallel execution is enabled: false
2015-02-10 22:48:54,624 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [second]  starting jobs: 2
2015-02-10 22:48:54,624 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [second]  allocating threads: 1
2015-02-10 22:48:54,625 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] starting step: (1/2)
2015-02-10 22:48:54,626 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:54,627 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:54,662 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:54,681 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:54,696 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local2033889522_0020
2015-02-10 22:48:54,715 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity2033889522/.staging/job_local2033889522_0020/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:54,716 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity2033889522/.staging/job_local2033889522_0020/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:54,764 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local2033889522_0020/job_local2033889522_0020.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:54,765 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local2033889522_0020/job_local2033889522_0020.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:54,766 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:54,766 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:54,767 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:54,767 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] submitted hadoop job: job_local2033889522_0020
2015-02-10 22:48:54,767 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] tracking url: http://localhost:8080/
2015-02-10 22:48:54,769 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:54,769 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local2033889522_0020_m_000000_0
2015-02-10 22:48:54,771 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:54,772 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:54,773 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1330c35c
2015-02-10 22:48:54,774 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:54,774 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:54,794 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:54,794 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:54,794 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:54,794 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:54,795 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:54,796 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:54,797 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:54,807 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:54,808 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(first)[by:[{1}:'char']]
2015-02-10 22:48:54,808 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:54,812 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:54,813 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 206, total: 750, max: 750
2015-02-10 22:48:54,814 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:54,814 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 206, total: 750, max: 750
2015-02-10 22:48:54,815 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:54,815 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 206, total: 750, max: 750
2015-02-10 22:48:54,815 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:54,815 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 206, total: 750, max: 750
2015-02-10 22:48:54,817 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:54,817 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:54,817 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:54,817 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 22:48:54,817 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:54,821 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:54,823 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local2033889522_0020_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:54,825 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:54,825 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2033889522_0020_m_000000_0' done.
2015-02-10 22:48:54,825 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local2033889522_0020_m_000000_0
2015-02-10 22:48:54,825 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:54,826 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:54,826 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local2033889522_0020_r_000000_0
2015-02-10 22:48:54,829 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:54,830 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@17ccad9c
2015-02-10 22:48:54,832 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551098752, maxSingleShuffleLimit=137774688, mergeThreshold=363725184, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:54,832 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local2033889522_0020_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:54,834 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#16 about to shuffle output of map attempt_local2033889522_0020_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 22:48:54,835 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local2033889522_0020_m_000000_0
2015-02-10 22:48:54,835 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 22:48:54,837 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:54,838 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:54,838 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:54,840 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:54,840 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:54,842 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:54,842 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 22:48:54,842 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:54,843 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:54,843 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:54,844 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:54,845 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:54,845 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:54,853 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(first)[by:[{1}:'char']]
2015-02-10 22:48:54,854 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][7042273824/first/]
2015-02-10 22:48:54,863 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local2033889522_0020_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:54,864 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:54,865 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local2033889522_0020_r_000000_0 is allowed to commit now
2015-02-10 22:48:54,865 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local2033889522_0020_r_000000_0' to file:/tmp/hadoop-teamcity/7042273824_first_F9B9801D789A48268AB65870D7CE7FC8/_temporary/0/task_local2033889522_0020_r_000000
2015-02-10 22:48:54,866 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:54,866 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2033889522_0020_r_000000_0' done.
2015-02-10 22:48:54,867 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local2033889522_0020_r_000000_0
2015-02-10 22:48:54,867 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:54,885 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] starting step: (2/2) ...form/samesourcemergecount
2015-02-10 22:48:54,886 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:54,887 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:54,923 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:54,942 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:54,952 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:54,952 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:54,958 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1981796583_0021
2015-02-10 22:48:54,975 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1981796583/.staging/job_local1981796583_0021/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:54,976 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1981796583/.staging/job_local1981796583_0021/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:55,024 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1981796583_0021/job_local1981796583_0021.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:55,025 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1981796583_0021/job_local1981796583_0021.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:55,025 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:55,026 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] submitted hadoop job: job_local1981796583_0021
2015-02-10 22:48:55,026 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:55,026 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] tracking url: http://localhost:8080/
2015-02-10 22:48:55,026 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:55,029 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:55,029 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1981796583_0021_m_000000_0
2015-02-10 22:48:55,031 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:55,032 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/7042273824_first_F9B9801D789A48268AB65870D7CE7FC8/part-00000
2015-02-10 22:48:55,032 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1367dca
2015-02-10 22:48:55,034 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:55,034 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:55,061 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:55,062 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:55,062 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:55,062 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:55,062 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:55,063 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:55,063 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:55,074 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][7042273824/first/]
2015-02-10 22:48:55,074 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(second)[by:[{1}:'char']]
2015-02-10 22:48:55,075 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:55,076 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:55,076 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 750, max: 750
2015-02-10 22:48:55,077 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:55,077 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 750, max: 750
2015-02-10 22:48:55,079 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:55,079 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:55,079 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:55,079 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 55; bufvoid = 104857600
2015-02-10 22:48:55,079 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:48:55,082 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:55,083 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1981796583_0021_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:55,085 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/7042273824_first_F9B9801D789A48268AB65870D7CE7FC8/part-00000:0+160
2015-02-10 22:48:55,086 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1981796583_0021_m_000000_0' done.
2015-02-10 22:48:55,086 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1981796583_0021_m_000000_0
2015-02-10 22:48:55,086 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:55,087 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:55,087 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1981796583_0021_r_000000_0
2015-02-10 22:48:55,088 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:55,088 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@3f4daa36
2015-02-10 22:48:55,089 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551098752, maxSingleShuffleLimit=137774688, mergeThreshold=363725184, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:55,090 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1981796583_0021_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:55,092 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#17 about to shuffle output of map attempt_local1981796583_0021_m_000000_0 decomp: 67 len: 71 to MEMORY
2015-02-10 22:48:55,092 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 67 bytes from map-output for attempt_local1981796583_0021_m_000000_0
2015-02-10 22:48:55,092 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 67, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->67
2015-02-10 22:48:55,093 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:55,095 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:55,095 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:55,097 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:55,098 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 22:48:55,099 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 67 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:55,100 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 71 bytes from disk
2015-02-10 22:48:55,100 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:55,101 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:55,101 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 22:48:55,102 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:55,102 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:55,103 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:55,113 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(second)[by:[{1}:'char']]
2015-02-10 22:48:55,113 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount"]
2015-02-10 22:48:55,124 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1981796583_0021_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:55,125 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:55,125 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1981796583_0021_r_000000_0 is allowed to commit now
2015-02-10 22:48:55,126 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1981796583_0021_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount/_temporary/0/task_local1981796583_0021_r_000000
2015-02-10 22:48:55,127 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:55,128 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1981796583_0021_r_000000_0' done.
2015-02-10 22:48:55,128 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1981796583_0021_r_000000_0
2015-02-10 22:48:55,128 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:55,153 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount/_temporary
2015-02-10 22:48:55,182 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:55,213 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:55,215 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount
2015-02-10 22:48:55,235 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:55,236 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:55,243 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:55,245 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:55,245 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:55,250 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average] starting
2015-02-10 22:48:55,250 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:55,250 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  sink: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average"]
2015-02-10 22:48:55,251 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  parallel execution is enabled: false
2015-02-10 22:48:55,251 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  starting jobs: 1
2015-02-10 22:48:55,251 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  allocating threads: 1
2015-02-10 22:48:55,251 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] starting step: (1/1) ...lyhelpersplatform/average
2015-02-10 22:48:55,252 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:55,253 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:55,279 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:55,301 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:55,317 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local273492952_0022
2015-02-10 22:48:55,334 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity273492952/.staging/job_local273492952_0022/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:55,336 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity273492952/.staging/job_local273492952_0022/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:55,382 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local273492952_0022/job_local273492952_0022.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:55,384 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local273492952_0022/job_local273492952_0022.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:55,384 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:55,385 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:55,385 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:55,385 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] submitted hadoop job: job_local273492952_0022
2015-02-10 22:48:55,386 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] tracking url: http://localhost:8080/
2015-02-10 22:48:55,388 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:55,388 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local273492952_0022_m_000000_0
2015-02-10 22:48:55,390 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:55,392 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:55,392 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@62892cc5
2015-02-10 22:48:55,394 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:55,395 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:55,423 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:55,423 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:55,423 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:55,423 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:55,424 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:55,425 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:55,425 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:55,434 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:55,435 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average)[by:[{1}:'char']]
2015-02-10 22:48:55,436 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:55,437 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:55,438 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 751, max: 751
2015-02-10 22:48:55,439 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:55,439 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 751, max: 751
2015-02-10 22:48:55,440 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:55,440 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 751, max: 751
2015-02-10 22:48:55,441 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:55,441 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 751, max: 751
2015-02-10 22:48:55,442 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:55,443 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:55,443 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:55,443 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2015-02-10 22:48:55,444 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:55,447 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:55,449 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local273492952_0022_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:55,451 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:55,451 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local273492952_0022_m_000000_0' done.
2015-02-10 22:48:55,452 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local273492952_0022_m_000000_0
2015-02-10 22:48:55,452 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:55,452 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:55,453 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local273492952_0022_r_000000_0
2015-02-10 22:48:55,456 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:55,456 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@4005f23d
2015-02-10 22:48:55,457 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551328128, maxSingleShuffleLimit=137832032, mergeThreshold=363876576, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:55,457 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local273492952_0022_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:55,459 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#18 about to shuffle output of map attempt_local273492952_0022_m_000000_0 decomp: 200 len: 204 to MEMORY
2015-02-10 22:48:55,460 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local273492952_0022_m_000000_0
2015-02-10 22:48:55,460 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->200
2015-02-10 22:48:55,461 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:55,462 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:55,462 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:55,464 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:55,464 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2015-02-10 22:48:55,466 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 200 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:55,466 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 204 bytes from disk
2015-02-10 22:48:55,466 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:55,466 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:55,467 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2015-02-10 22:48:55,467 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:55,467 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:55,468 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:55,474 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(average)[by:[{1}:'char']]
2015-02-10 22:48:55,474 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average"]
2015-02-10 22:48:55,485 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local273492952_0022_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:55,486 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:55,486 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local273492952_0022_r_000000_0 is allowed to commit now
2015-02-10 22:48:55,487 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local273492952_0022_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average/_temporary/0/task_local273492952_0022_r_000000
2015-02-10 22:48:55,488 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:55,488 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local273492952_0022_r_000000_0' done.
2015-02-10 22:48:55,489 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local273492952_0022_r_000000_0
2015-02-10 22:48:55,489 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:55,512 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average/_temporary
2015-02-10 22:48:55,543 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:55,577 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:55,580 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/average
2015-02-10 22:48:55,602 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:55,602 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:55,609 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:55,612 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:55,613 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:55,617 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting
2015-02-10 22:48:55,618 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:55,618 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  sink: Hfs["TextDelimited[['char', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety"]
2015-02-10 22:48:55,618 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  parallel execution is enabled: false
2015-02-10 22:48:55,618 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  starting jobs: 1
2015-02-10 22:48:55,618 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  allocating threads: 1
2015-02-10 22:48:55,619 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...splatform/maxbynullsafety
2015-02-10 22:48:55,620 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:55,621 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:55,645 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:55,664 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:55,679 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1009360269_0023
2015-02-10 22:48:55,696 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1009360269/.staging/job_local1009360269_0023/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:55,697 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1009360269/.staging/job_local1009360269_0023/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:55,746 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1009360269_0023/job_local1009360269_0023.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:55,747 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1009360269_0023/job_local1009360269_0023.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:55,748 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:55,748 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local1009360269_0023
2015-02-10 22:48:55,748 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:55,749 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:55,748 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2015-02-10 22:48:55,758 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:55,758 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1009360269_0023_m_000000_0
2015-02-10 22:48:55,759 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:55,761 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:55,761 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7d4720ef
2015-02-10 22:48:55,762 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:55,762 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:55,790 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:55,791 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:55,791 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:55,791 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:55,791 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:55,793 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:55,793 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:55,800 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:55,801 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'char']]
2015-02-10 22:48:55,801 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:55,803 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:55,803 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 751, max: 751
2015-02-10 22:48:55,804 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:55,805 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 751, max: 751
2015-02-10 22:48:55,806 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:55,806 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 751, max: 751
2015-02-10 22:48:55,807 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:55,807 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 751, max: 751
2015-02-10 22:48:55,808 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:55,809 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:55,809 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:55,809 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 98; bufvoid = 104857600
2015-02-10 22:48:55,810 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:55,813 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:55,815 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1009360269_0023_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:55,817 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:55,817 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1009360269_0023_m_000000_0' done.
2015-02-10 22:48:55,818 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1009360269_0023_m_000000_0
2015-02-10 22:48:55,819 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:55,820 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:55,820 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1009360269_0023_r_000000_0
2015-02-10 22:48:55,821 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:55,822 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@47a77e1e
2015-02-10 22:48:55,822 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551374016, maxSingleShuffleLimit=137843504, mergeThreshold=363906880, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:55,823 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1009360269_0023_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:55,824 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#19 about to shuffle output of map attempt_local1009360269_0023_m_000000_0 decomp: 118 len: 122 to MEMORY
2015-02-10 22:48:55,825 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 118 bytes from map-output for attempt_local1009360269_0023_m_000000_0
2015-02-10 22:48:55,825 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 118, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->118
2015-02-10 22:48:55,826 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:55,827 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:55,828 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:55,829 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:55,830 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 109 bytes
2015-02-10 22:48:55,831 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 118 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:55,832 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 122 bytes from disk
2015-02-10 22:48:55,832 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:55,832 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:55,832 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 109 bytes
2015-02-10 22:48:55,833 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:55,833 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:55,833 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:55,841 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'char']]
2015-02-10 22:48:55,841 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety"]
2015-02-10 22:48:55,852 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1009360269_0023_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:55,854 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:55,854 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1009360269_0023_r_000000_0 is allowed to commit now
2015-02-10 22:48:55,855 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1009360269_0023_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety/_temporary/0/task_local1009360269_0023_r_000000
2015-02-10 22:48:55,856 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:55,856 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1009360269_0023_r_000000_0' done.
2015-02-10 22:48:55,856 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1009360269_0023_r_000000_0
2015-02-10 22:48:55,856 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:55,880 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety/_temporary
2015-02-10 22:48:55,910 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:55,940 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:55,942 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety
2015-02-10 22:48:55,963 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:55,963 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:55,970 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:55,973 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:55,974 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:55,980 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] starting
2015-02-10 22:48:55,980 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:55,980 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  sink: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls"]
2015-02-10 22:48:55,980 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  parallel execution is enabled: false
2015-02-10 22:48:55,982 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  starting jobs: 1
2015-02-10 22:48:55,982 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  allocating threads: 1
2015-02-10 22:48:55,983 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] starting step: (1/1) ...yhelpersplatform/sumnulls
2015-02-10 22:48:55,984 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:55,985 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:56,008 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:56,026 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:56,041 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1796135208_0024
2015-02-10 22:48:56,058 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1796135208/.staging/job_local1796135208_0024/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:56,059 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1796135208/.staging/job_local1796135208_0024/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:56,108 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1796135208_0024/job_local1796135208_0024.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:56,109 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1796135208_0024/job_local1796135208_0024.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:56,109 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:56,110 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] submitted hadoop job: job_local1796135208_0024
2015-02-10 22:48:56,110 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:56,110 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] tracking url: http://localhost:8080/
2015-02-10 22:48:56,110 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:56,115 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:56,115 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1796135208_0024_m_000000_0
2015-02-10 22:48:56,117 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:56,118 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:56,118 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1d272e52
2015-02-10 22:48:56,120 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:56,120 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:56,149 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:56,149 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:56,149 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:56,149 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:56,149 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:56,151 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:56,151 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:56,160 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:56,160 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum)[by:[{1}:'char']]
2015-02-10 22:48:56,161 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:56,170 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:56,170 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 751, max: 751
2015-02-10 22:48:56,171 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:56,171 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 751, max: 751
2015-02-10 22:48:56,172 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:56,172 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 751, max: 751
2015-02-10 22:48:56,173 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:56,173 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 751, max: 751
2015-02-10 22:48:56,174 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:56,174 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:56,174 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:56,175 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 96; bufvoid = 104857600
2015-02-10 22:48:56,175 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:56,178 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:56,180 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1796135208_0024_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:56,182 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:56,182 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1796135208_0024_m_000000_0' done.
2015-02-10 22:48:56,182 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1796135208_0024_m_000000_0
2015-02-10 22:48:56,183 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:56,183 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:56,184 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1796135208_0024_r_000000_0
2015-02-10 22:48:56,185 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:56,186 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@291972f8
2015-02-10 22:48:56,186 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551603392, maxSingleShuffleLimit=137900848, mergeThreshold=364058240, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:56,187 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1796135208_0024_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:56,190 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#20 about to shuffle output of map attempt_local1796135208_0024_m_000000_0 decomp: 116 len: 120 to MEMORY
2015-02-10 22:48:56,190 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 116 bytes from map-output for attempt_local1796135208_0024_m_000000_0
2015-02-10 22:48:56,190 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 116, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->116
2015-02-10 22:48:56,191 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:56,192 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:56,193 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:56,195 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:56,195 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 107 bytes
2015-02-10 22:48:56,197 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 116 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:56,197 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 120 bytes from disk
2015-02-10 22:48:56,197 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:56,197 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:56,197 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 107 bytes
2015-02-10 22:48:56,198 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:56,198 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:56,199 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:56,204 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(sum)[by:[{1}:'char']]
2015-02-10 22:48:56,205 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls"]
2015-02-10 22:48:56,215 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1796135208_0024_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:56,216 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:56,216 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1796135208_0024_r_000000_0 is allowed to commit now
2015-02-10 22:48:56,217 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1796135208_0024_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls/_temporary/0/task_local1796135208_0024_r_000000
2015-02-10 22:48:56,218 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:56,218 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1796135208_0024_r_000000_0' done.
2015-02-10 22:48:56,218 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1796135208_0024_r_000000_0
2015-02-10 22:48:56,219 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:56,236 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls/_temporary
2015-02-10 22:48:56,267 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:56,298 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:56,300 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/sumnulls
2015-02-10 22:48:56,321 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:56,322 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:56,328 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:56,330 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:56,331 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:56,335 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting
2015-02-10 22:48:56,335 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:56,336 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  sink: Hfs["TextDelimited[['count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall"]
2015-02-10 22:48:56,336 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  parallel execution is enabled: false
2015-02-10 22:48:56,336 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  starting jobs: 1
2015-02-10 22:48:56,336 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  allocating threads: 1
2015-02-10 22:48:56,337 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...yhelpersplatform/countall
2015-02-10 22:48:56,337 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:56,338 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:56,363 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:56,381 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:56,398 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1793719809_0025
2015-02-10 22:48:56,416 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1793719809/.staging/job_local1793719809_0025/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:56,417 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1793719809/.staging/job_local1793719809_0025/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:56,466 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1793719809_0025/job_local1793719809_0025.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:56,467 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1793719809_0025/job_local1793719809_0025.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:56,467 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:56,468 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local1793719809_0025
2015-02-10 22:48:56,468 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:56,468 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:56,468 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-10 22:48:56,471 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:56,471 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1793719809_0025_m_000000_0
2015-02-10 22:48:56,472 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:56,474 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:56,474 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@30e6ace7
2015-02-10 22:48:56,475 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:56,475 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:56,503 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:56,504 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:56,504 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:56,504 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:56,504 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:56,505 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:56,505 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:56,513 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:56,513 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{?}:NONE]]
2015-02-10 22:48:56,514 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:56,517 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:56,517 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:56,517 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:56,518 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 4; bufvoid = 104857600
2015-02-10 22:48:56,519 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214396(104857584); length = 1/6553600
2015-02-10 22:48:56,523 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:56,525 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1793719809_0025_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:56,528 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:56,528 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1793719809_0025_m_000000_0' done.
2015-02-10 22:48:56,528 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1793719809_0025_m_000000_0
2015-02-10 22:48:56,529 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:56,529 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:56,529 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1793719809_0025_r_000000_0
2015-02-10 22:48:56,532 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:56,533 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@65e7c41f
2015-02-10 22:48:56,533 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551649280, maxSingleShuffleLimit=137912320, mergeThreshold=364088544, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:56,534 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1793719809_0025_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:56,536 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#21 about to shuffle output of map attempt_local1793719809_0025_m_000000_0 decomp: 8 len: 12 to MEMORY
2015-02-10 22:48:56,536 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 8 bytes from map-output for attempt_local1793719809_0025_m_000000_0
2015-02-10 22:48:56,536 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 8, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->8
2015-02-10 22:48:56,538 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:56,539 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:56,540 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:56,542 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:56,542 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 5 bytes
2015-02-10 22:48:56,543 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 8 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:56,544 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 12 bytes from disk
2015-02-10 22:48:56,544 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:56,544 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:56,544 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 5 bytes
2015-02-10 22:48:56,545 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:56,545 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:56,546 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:56,553 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{?}:NONE]]
2015-02-10 22:48:56,553 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall"]
2015-02-10 22:48:56,562 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1793719809_0025_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:56,563 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:56,564 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1793719809_0025_r_000000_0 is allowed to commit now
2015-02-10 22:48:56,564 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1793719809_0025_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall/_temporary/0/task_local1793719809_0025_r_000000
2015-02-10 22:48:56,565 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:56,566 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1793719809_0025_r_000000_0' done.
2015-02-10 22:48:56,566 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1793719809_0025_r_000000_0
2015-02-10 22:48:56,566 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:56,581 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall/_temporary
2015-02-10 22:48:56,615 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:56,651 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:56,653 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/countall
2015-02-10 22:48:56,677 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:56,678 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:56,687 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:56,688 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:56,689 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:56,694 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name] starting
2015-02-10 22:48:56,695 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:56,695 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average', 'average2', 'first']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi"]
2015-02-10 22:48:56,696 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  parallel execution is enabled: false
2015-02-10 22:48:56,696 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  starting jobs: 1
2015-02-10 22:48:56,696 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  allocating threads: 1
2015-02-10 22:48:56,697 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] starting step: (1/1) ...mblyhelpersplatform/multi
2015-02-10 22:48:56,698 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:56,699 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:56,723 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:56,744 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:56,759 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local105142053_0026
2015-02-10 22:48:56,777 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity105142053/.staging/job_local105142053_0026/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:56,778 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity105142053/.staging/job_local105142053_0026/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:56,824 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local105142053_0026/job_local105142053_0026.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:56,826 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local105142053_0026/job_local105142053_0026.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:56,826 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:56,827 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:56,827 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:56,827 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] submitted hadoop job: job_local105142053_0026
2015-02-10 22:48:56,828 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] tracking url: http://localhost:8080/
2015-02-10 22:48:56,830 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:56,830 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local105142053_0026_m_000000_0
2015-02-10 22:48:56,831 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:56,832 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:56,833 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1ea44130
2015-02-10 22:48:56,834 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:56,834 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:56,862 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:56,862 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:56,862 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:56,862 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:56,862 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:56,864 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:56,864 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:56,874 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:56,874 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2015-02-10 22:48:56,874 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:56,876 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:56,876 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 751, max: 751
2015-02-10 22:48:56,878 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:56,878 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 751, max: 751
2015-02-10 22:48:56,879 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:56,879 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 751, max: 751
2015-02-10 22:48:56,879 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:56,880 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 751, max: 751
2015-02-10 22:48:56,881 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:56,881 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:56,881 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:56,881 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 432; bufvoid = 104857600
2015-02-10 22:48:56,882 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:56,885 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:56,886 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local105142053_0026_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:56,888 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:56,888 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local105142053_0026_m_000000_0' done.
2015-02-10 22:48:56,889 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local105142053_0026_m_000000_0
2015-02-10 22:48:56,889 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:56,889 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:56,889 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local105142053_0026_r_000000_0
2015-02-10 22:48:56,892 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:56,892 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@3997ebf6
2015-02-10 22:48:56,893 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551878656, maxSingleShuffleLimit=137969664, mergeThreshold=364239936, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:56,894 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local105142053_0026_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:56,896 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#22 about to shuffle output of map attempt_local105142053_0026_m_000000_0 decomp: 452 len: 456 to MEMORY
2015-02-10 22:48:56,897 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 452 bytes from map-output for attempt_local105142053_0026_m_000000_0
2015-02-10 22:48:56,897 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 452, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->452
2015-02-10 22:48:56,898 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:56,899 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:56,899 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:56,901 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:56,901 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 436 bytes
2015-02-10 22:48:56,903 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 452 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:56,903 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 456 bytes from disk
2015-02-10 22:48:56,903 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:56,903 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:56,904 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 436 bytes
2015-02-10 22:48:56,904 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:56,905 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:56,905 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:56,914 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(name)[by:[{1}:'char']]
2015-02-10 22:48:56,914 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average', 'average2', 'first']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi"]
2015-02-10 22:48:56,926 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local105142053_0026_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:56,927 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:56,927 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local105142053_0026_r_000000_0 is allowed to commit now
2015-02-10 22:48:56,928 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local105142053_0026_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi/_temporary/0/task_local105142053_0026_r_000000
2015-02-10 22:48:56,929 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:56,929 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local105142053_0026_r_000000_0' done.
2015-02-10 22:48:56,929 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local105142053_0026_r_000000_0
2015-02-10 22:48:56,929 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:56,950 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi/_temporary
2015-02-10 22:48:56,979 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:57,013 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:57,014 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/multi
2015-02-10 22:48:57,033 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:57,033 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:57,038 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:57,040 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:57,040 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:57,045 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 22:48:57,045 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:57,045 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  sink: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow"]
2015-02-10 22:48:57,045 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  parallel execution is enabled: false
2015-02-10 22:48:57,046 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  starting jobs: 1
2015-02-10 22:48:57,046 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  allocating threads: 1
2015-02-10 22:48:57,047 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...persplatform/retainnarrow
2015-02-10 22:48:57,047 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:57,049 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:57,075 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:57,094 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:57,115 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1476339018_0027
2015-02-10 22:48:57,134 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1476339018/.staging/job_local1476339018_0027/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:57,136 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1476339018/.staging/job_local1476339018_0027/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:57,188 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1476339018_0027/job_local1476339018_0027.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:57,189 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1476339018_0027/job_local1476339018_0027.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:57,190 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:57,191 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1476339018_0027
2015-02-10 22:48:57,191 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:57,191 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 22:48:57,191 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:57,194 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:57,194 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1476339018_0027_m_000000_0
2015-02-10 22:48:57,195 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:57,197 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt
2015-02-10 22:48:57,198 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@17a220d7
2015-02-10 22:48:57,199 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 22:48:57,207 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:57,207 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:57,212 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:57,212 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow"]
2015-02-10 22:48:57,215 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:57,215 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1476339018_0027_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:57,217 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:57,217 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1476339018_0027_m_000000_0 is allowed to commit now
2015-02-10 22:48:57,218 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1476339018_0027_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow/_temporary/0/task_local1476339018_0027_m_000000
2015-02-10 22:48:57,219 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt:0+19
2015-02-10 22:48:57,219 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1476339018_0027_m_000000_0' done.
2015-02-10 22:48:57,219 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1476339018_0027_m_000000_0
2015-02-10 22:48:57,219 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:57,238 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow/_temporary
2015-02-10 22:48:57,270 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:57,272 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/retainnarrow
2015-02-10 22:48:57,292 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:57,293 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:57,299 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:57,301 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:57,302 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:57,305 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 22:48:57,306 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:57,306 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  sink: Hfs["TextLine[['item']->['char', 'item']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow"]
2015-02-10 22:48:57,306 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  parallel execution is enabled: false
2015-02-10 22:48:57,306 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  starting jobs: 1
2015-02-10 22:48:57,306 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  allocating threads: 1
2015-02-10 22:48:57,307 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...persplatform/renamenarrow
2015-02-10 22:48:57,308 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:57,309 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:57,335 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:57,353 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:57,369 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1246039426_0028
2015-02-10 22:48:57,386 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1246039426/.staging/job_local1246039426_0028/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:57,388 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1246039426/.staging/job_local1246039426_0028/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:57,439 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1246039426_0028/job_local1246039426_0028.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:57,441 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1246039426_0028/job_local1246039426_0028.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:57,441 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:57,442 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1246039426_0028
2015-02-10 22:48:57,442 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 22:48:57,443 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:57,443 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:57,446 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:57,446 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1246039426_0028_m_000000_0
2015-02-10 22:48:57,448 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:57,449 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt
2015-02-10 22:48:57,450 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@bc0fc25
2015-02-10 22:48:57,451 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 22:48:57,459 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:57,459 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:57,463 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:57,463 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['item']->['char', 'item']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow"]
2015-02-10 22:48:57,466 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:57,466 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1246039426_0028_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:57,468 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:57,468 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1246039426_0028_m_000000_0 is allowed to commit now
2015-02-10 22:48:57,468 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1246039426_0028_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow/_temporary/0/task_local1246039426_0028_m_000000
2015-02-10 22:48:57,469 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt:0+19
2015-02-10 22:48:57,470 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1246039426_0028_m_000000_0' done.
2015-02-10 22:48:57,470 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1246039426_0028_m_000000_0
2015-02-10 22:48:57,470 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:57,486 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow/_temporary
2015-02-10 22:48:57,518 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:57,521 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/renamenarrow
2015-02-10 22:48:57,543 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:57,544 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:57,551 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:57,552 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:57,553 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:57,559 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average] starting
2015-02-10 22:48:57,559 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:57,559 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  sink: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull"]
2015-02-10 22:48:57,560 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  parallel execution is enabled: false
2015-02-10 22:48:57,560 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  starting jobs: 1
2015-02-10 22:48:57,560 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  allocating threads: 1
2015-02-10 22:48:57,561 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] starting step: (1/1) ...lpersplatform/averagenull
2015-02-10 22:48:57,562 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:57,563 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:57,587 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:57,605 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:57,630 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local353443111_0029
2015-02-10 22:48:57,650 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity353443111/.staging/job_local353443111_0029/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:57,651 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity353443111/.staging/job_local353443111_0029/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:57,699 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local353443111_0029/job_local353443111_0029.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:57,700 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local353443111_0029/job_local353443111_0029.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:57,701 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:57,702 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] submitted hadoop job: job_local353443111_0029
2015-02-10 22:48:57,702 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:57,702 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] tracking url: http://localhost:8080/
2015-02-10 22:48:57,702 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:57,705 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:57,705 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local353443111_0029_m_000000_0
2015-02-10 22:48:57,706 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:57,708 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:57,708 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5b606e1f
2015-02-10 22:48:57,709 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:57,709 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:57,729 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:57,729 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:57,730 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:57,730 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:57,730 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:57,731 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:57,731 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:57,739 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:57,739 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average)[by:[{1}:'char']]
2015-02-10 22:48:57,739 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:57,743 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:57,744 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 221, total: 751, max: 751
2015-02-10 22:48:57,745 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:57,745 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 221, total: 751, max: 751
2015-02-10 22:48:57,746 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:57,746 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 221, total: 751, max: 751
2015-02-10 22:48:57,746 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:57,747 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 221, total: 751, max: 751
2015-02-10 22:48:57,748 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:57,748 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:57,748 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:57,748 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2015-02-10 22:48:57,748 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:57,751 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:57,753 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local353443111_0029_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:57,755 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:57,755 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local353443111_0029_m_000000_0' done.
2015-02-10 22:48:57,755 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local353443111_0029_m_000000_0
2015-02-10 22:48:57,756 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:57,756 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:57,756 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local353443111_0029_r_000000_0
2015-02-10 22:48:57,758 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:57,758 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@1bb66a6b
2015-02-10 22:48:57,759 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551924544, maxSingleShuffleLimit=137981136, mergeThreshold=364270208, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:57,759 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local353443111_0029_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:57,763 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#23 about to shuffle output of map attempt_local353443111_0029_m_000000_0 decomp: 200 len: 204 to MEMORY
2015-02-10 22:48:57,763 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local353443111_0029_m_000000_0
2015-02-10 22:48:57,764 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->200
2015-02-10 22:48:57,765 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:57,765 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:57,766 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:57,768 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:57,768 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2015-02-10 22:48:57,769 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 200 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:57,769 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 204 bytes from disk
2015-02-10 22:48:57,770 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:57,770 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:57,770 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2015-02-10 22:48:57,771 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:57,771 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:57,771 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:57,778 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(average)[by:[{1}:'char']]
2015-02-10 22:48:57,778 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull"]
2015-02-10 22:48:57,788 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local353443111_0029_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:57,790 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:57,790 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local353443111_0029_r_000000_0 is allowed to commit now
2015-02-10 22:48:57,790 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local353443111_0029_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull/_temporary/0/task_local353443111_0029_r_000000
2015-02-10 22:48:57,792 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:57,792 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local353443111_0029_r_000000_0' done.
2015-02-10 22:48:57,792 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local353443111_0029_r_000000_0
2015-02-10 22:48:57,792 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:57,809 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull/_temporary
2015-02-10 22:48:57,839 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:57,869 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:57,871 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/averagenull
2015-02-10 22:48:57,890 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:57,891 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:57,896 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:57,898 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:57,899 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:57,902 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] starting
2015-02-10 22:48:57,903 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:57,903 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  sink: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce"]
2015-02-10 22:48:57,903 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  parallel execution is enabled: false
2015-02-10 22:48:57,903 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  starting jobs: 1
2015-02-10 22:48:57,903 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  allocating threads: 1
2015-02-10 22:48:57,904 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] starting step: (1/1) ...blyhelpersplatform/coerce
2015-02-10 22:48:57,904 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:57,905 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:57,928 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:57,946 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:57,961 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local2036586552_0030
2015-02-10 22:48:57,978 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity2036586552/.staging/job_local2036586552_0030/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:57,979 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity2036586552/.staging/job_local2036586552_0030/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:58,026 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local2036586552_0030/job_local2036586552_0030.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:58,027 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local2036586552_0030/job_local2036586552_0030.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:58,028 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:58,028 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] submitted hadoop job: job_local2036586552_0030
2015-02-10 22:48:58,028 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] tracking url: http://localhost:8080/
2015-02-10 22:48:58,030 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:58,030 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:58,034 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:58,034 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local2036586552_0030_m_000000_0
2015-02-10 22:48:58,035 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:58,036 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt
2015-02-10 22:48:58,037 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@77dbc064
2015-02-10 22:48:58,038 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 22:48:58,044 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:58,044 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:58,048 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:58,049 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce"]
2015-02-10 22:48:58,051 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:58,051 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local2036586552_0030_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:58,052 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:58,053 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local2036586552_0030_m_000000_0 is allowed to commit now
2015-02-10 22:48:58,053 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local2036586552_0030_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce/_temporary/0/task_local2036586552_0030_m_000000
2015-02-10 22:48:58,054 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt:0+19
2015-02-10 22:48:58,054 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2036586552_0030_m_000000_0' done.
2015-02-10 22:48:58,054 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local2036586552_0030_m_000000_0
2015-02-10 22:48:58,055 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:58,071 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce/_temporary
2015-02-10 22:48:58,101 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:58,103 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/coerce
2015-02-10 22:48:58,122 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:58,122 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:58,128 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:58,130 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:58,130 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:58,134 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first] starting
2015-02-10 22:48:58,135 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  source: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs+rhs-cross-rev.txt"]
2015-02-10 22:48:58,135 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  sink: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator"]
2015-02-10 22:48:58,135 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  parallel execution is enabled: false
2015-02-10 22:48:58,135 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  starting jobs: 1
2015-02-10 22:48:58,135 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  allocating threads: 1
2015-02-10 22:48:58,136 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] starting step: (1/1) ...tnfieldswithoutcomparator
2015-02-10 22:48:58,137 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:58,138 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:58,162 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:58,181 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:58,197 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1994969962_0031
2015-02-10 22:48:58,215 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1994969962/.staging/job_local1994969962_0031/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:58,216 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1994969962/.staging/job_local1994969962_0031/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:58,267 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1994969962_0031/job_local1994969962_0031.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:58,268 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1994969962_0031/job_local1994969962_0031.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:58,269 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:58,269 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] submitted hadoop job: job_local1994969962_0031
2015-02-10 22:48:58,270 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:58,270 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] tracking url: http://localhost:8080/
2015-02-10 22:48:58,270 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:58,273 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:58,273 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1994969962_0031_m_000000_0
2015-02-10 22:48:58,275 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:58,276 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs+rhs-cross-rev.txt
2015-02-10 22:48:58,277 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@504be958
2015-02-10 22:48:58,278 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:58,278 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:58,311 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:58,311 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:58,311 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:58,311 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:58,311 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:58,313 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:58,313 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:58,318 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs+rhs-cross-rev.txt"]
2015-02-10 22:48:58,318 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(first)[by:[{1}:'num']]
2015-02-10 22:48:58,319 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:58,321 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:58,321 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 752, max: 752
2015-02-10 22:48:58,323 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:58,323 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 752, max: 752
2015-02-10 22:48:58,325 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:58,325 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:58,325 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:58,325 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 105; bufvoid = 104857600
2015-02-10 22:48:58,325 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:48:58,328 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:58,329 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1994969962_0031_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:58,331 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs+rhs-cross-rev.txt:0+221
2015-02-10 22:48:58,331 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1994969962_0031_m_000000_0' done.
2015-02-10 22:48:58,331 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1994969962_0031_m_000000_0
2015-02-10 22:48:58,332 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:58,332 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:58,332 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1994969962_0031_r_000000_0
2015-02-10 22:48:58,334 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:58,334 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@3aa7ff2e
2015-02-10 22:48:58,335 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552153920, maxSingleShuffleLimit=138038480, mergeThreshold=364421600, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:58,335 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1994969962_0031_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:58,337 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#24 about to shuffle output of map attempt_local1994969962_0031_m_000000_0 decomp: 117 len: 121 to MEMORY
2015-02-10 22:48:58,338 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 117 bytes from map-output for attempt_local1994969962_0031_m_000000_0
2015-02-10 22:48:58,338 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 117, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->117
2015-02-10 22:48:58,340 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:58,341 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:58,341 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:58,343 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:58,343 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 108 bytes
2015-02-10 22:48:58,344 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 117 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:58,345 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 121 bytes from disk
2015-02-10 22:48:58,345 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:58,345 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:58,345 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 108 bytes
2015-02-10 22:48:58,346 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:58,346 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:58,346 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:58,351 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(first)[by:[{1}:'num']]
2015-02-10 22:48:58,351 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator"]
2015-02-10 22:48:58,360 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1994969962_0031_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:58,362 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:58,362 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1994969962_0031_r_000000_0 is allowed to commit now
2015-02-10 22:48:58,363 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1994969962_0031_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator/_temporary/0/task_local1994969962_0031_r_000000
2015-02-10 22:48:58,364 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:58,364 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1994969962_0031_r_000000_0' done.
2015-02-10 22:48:58,364 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1994969962_0031_r_000000_0
2015-02-10 22:48:58,364 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:58,386 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator/_temporary
2015-02-10 22:48:58,415 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:58,417 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator
2015-02-10 22:48:58,442 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:58,442 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:58,469 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:58,471 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:58,493 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:58,498 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...] starting
2015-02-10 22:48:58,498 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:58,499 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...]  sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway"]
2015-02-10 22:48:58,499 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...]  parallel execution is enabled: false
2015-02-10 22:48:58,499 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...]  starting jobs: 5
2015-02-10 22:48:58,499 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...]  allocating threads: 1
2015-02-10 22:48:58,500 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (1/5)
2015-02-10 22:48:58,501 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:58,503 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:58,527 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:58,545 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:58,562 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1491073616_0032
2015-02-10 22:48:58,580 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1491073616/.staging/job_local1491073616_0032/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:58,581 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1491073616/.staging/job_local1491073616_0032/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:58,632 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1491073616_0032/job_local1491073616_0032.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:58,634 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1491073616_0032/job_local1491073616_0032.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:58,634 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:58,635 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1491073616_0032
2015-02-10 22:48:58,635 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-10 22:48:58,635 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:58,636 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:58,638 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:58,639 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1491073616_0032_m_000000_0
2015-02-10 22:48:58,641 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:58,643 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:58,643 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@586ca3ef
2015-02-10 22:48:58,645 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:58,645 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:58,680 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:58,680 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:58,680 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:58,680 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:58,680 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:58,682 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:58,682 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:58,688 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:58,688 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(source)[by:[{1}:'char']]
2015-02-10 22:48:58,689 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:58,690 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:58,690 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 752, max: 752
2015-02-10 22:48:58,691 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:58,692 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 752, max: 752
2015-02-10 22:48:58,692 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:58,693 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 752, max: 752
2015-02-10 22:48:58,693 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:58,694 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 752, max: 752
2015-02-10 22:48:58,695 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:58,695 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:58,695 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:58,695 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 22:48:58,696 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:58,699 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:58,701 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1491073616_0032_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:58,705 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:58,705 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1491073616_0032_m_000000_0' done.
2015-02-10 22:48:58,705 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1491073616_0032_m_000000_0
2015-02-10 22:48:58,706 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:58,707 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:58,707 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1491073616_0032_r_000000_0
2015-02-10 22:48:58,708 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:58,709 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@687090a8
2015-02-10 22:48:58,709 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552383296, maxSingleShuffleLimit=138095824, mergeThreshold=364572992, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:58,710 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1491073616_0032_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:58,713 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#25 about to shuffle output of map attempt_local1491073616_0032_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 22:48:58,713 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local1491073616_0032_m_000000_0
2015-02-10 22:48:58,713 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 22:48:58,714 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:58,715 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:58,715 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:58,717 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:58,717 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:58,718 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:58,719 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 22:48:58,719 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:58,719 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:58,720 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:58,721 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:58,721 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:58,721 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:58,727 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(source)[by:[{1}:'char']]
2015-02-10 22:48:58,727 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][5847483701/source/]
2015-02-10 22:48:58,737 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1491073616_0032_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:58,739 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:58,739 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1491073616_0032_r_000000_0 is allowed to commit now
2015-02-10 22:48:58,739 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1491073616_0032_r_000000_0' to file:/tmp/hadoop-teamcity/5847483701_source_77D070BA7154495FB0F0EE34ABC63BF8/_temporary/0/task_local1491073616_0032_r_000000
2015-02-10 22:48:58,741 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:58,741 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1491073616_0032_r_000000_0' done.
2015-02-10 22:48:58,741 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1491073616_0032_r_000000_0
2015-02-10 22:48:58,741 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:58,764 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (3/5)
2015-02-10 22:48:58,764 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:58,765 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:58,802 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:58,821 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:58,833 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:58,834 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:58,841 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1104083515_0033
2015-02-10 22:48:58,862 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1104083515/.staging/job_local1104083515_0033/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:58,863 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1104083515/.staging/job_local1104083515_0033/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:58,913 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1104083515_0033/job_local1104083515_0033.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:58,915 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1104083515_0033/job_local1104083515_0033.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:58,915 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:58,916 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1104083515_0033
2015-02-10 22:48:58,916 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-10 22:48:58,916 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:58,918 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:58,921 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:58,922 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1104083515_0033_m_000000_0
2015-02-10 22:48:58,924 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:58,926 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/5847483701_source_77D070BA7154495FB0F0EE34ABC63BF8/part-00000
2015-02-10 22:48:58,927 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4b5bc191
2015-02-10 22:48:58,928 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:58,929 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:58,961 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:58,962 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:58,962 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:58,962 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:58,963 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:58,964 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:58,964 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:58,970 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][5847483701/source/]
2015-02-10 22:48:58,971 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-lhs)[by:[{1}:'char']]
2015-02-10 22:48:58,972 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:58,973 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:58,974 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 752, max: 752
2015-02-10 22:48:58,974 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:58,975 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 752, max: 752
2015-02-10 22:48:58,976 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:58,977 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:58,977 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:58,977 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 55; bufvoid = 104857600
2015-02-10 22:48:58,977 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:48:59,033 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:59,036 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1104083515_0033_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:59,038 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/5847483701_source_77D070BA7154495FB0F0EE34ABC63BF8/part-00000:0+160
2015-02-10 22:48:59,038 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1104083515_0033_m_000000_0' done.
2015-02-10 22:48:59,038 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1104083515_0033_m_000000_0
2015-02-10 22:48:59,039 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:59,039 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:59,040 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1104083515_0033_r_000000_0
2015-02-10 22:48:59,041 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:59,041 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7e7d611f
2015-02-10 22:48:59,042 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552520896, maxSingleShuffleLimit=138130224, mergeThreshold=364663808, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:59,043 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1104083515_0033_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:59,045 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#26 about to shuffle output of map attempt_local1104083515_0033_m_000000_0 decomp: 67 len: 71 to MEMORY
2015-02-10 22:48:59,045 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 67 bytes from map-output for attempt_local1104083515_0033_m_000000_0
2015-02-10 22:48:59,045 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 67, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->67
2015-02-10 22:48:59,046 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:59,047 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:59,048 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:59,050 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:59,050 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 22:48:59,053 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 67 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:59,053 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 71 bytes from disk
2015-02-10 22:48:59,053 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:59,053 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:59,054 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 22:48:59,054 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:59,055 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:59,055 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:59,060 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count-lhs)[by:[{1}:'char']]
2015-02-10 22:48:59,060 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][152035209/count-lhs/]
2015-02-10 22:48:59,070 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1104083515_0033_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:59,071 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:59,071 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1104083515_0033_r_000000_0 is allowed to commit now
2015-02-10 22:48:59,072 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1104083515_0033_r_000000_0' to file:/tmp/hadoop-teamcity/152035209_count_lhs_856A3A2E025F419F972FB242029AC240/_temporary/0/task_local1104083515_0033_r_000000
2015-02-10 22:48:59,073 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:59,073 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1104083515_0033_r_000000_0' done.
2015-02-10 22:48:59,073 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1104083515_0033_r_000000_0
2015-02-10 22:48:59,073 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:59,086 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (4/5)
2015-02-10 22:48:59,086 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:59,087 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:59,125 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:59,147 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:59,159 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:59,159 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:59,164 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local725792397_0034
2015-02-10 22:48:59,183 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity725792397/.staging/job_local725792397_0034/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:59,184 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity725792397/.staging/job_local725792397_0034/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:59,231 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local725792397_0034/job_local725792397_0034.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:59,232 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local725792397_0034/job_local725792397_0034.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:59,233 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:59,233 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local725792397_0034
2015-02-10 22:48:59,234 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-10 22:48:59,234 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:59,234 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:59,236 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:59,237 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local725792397_0034_m_000000_0
2015-02-10 22:48:59,238 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:59,240 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/5847483701_source_77D070BA7154495FB0F0EE34ABC63BF8/part-00000
2015-02-10 22:48:59,240 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1aa6a14b
2015-02-10 22:48:59,242 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:59,242 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:59,274 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:59,274 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:59,275 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:59,275 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:59,275 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:59,277 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:59,277 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:59,282 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][5847483701/source/]
2015-02-10 22:48:59,283 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-rhs)[by:[{1}:'char']]
2015-02-10 22:48:59,283 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:59,285 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:59,285 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 752, max: 752
2015-02-10 22:48:59,286 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:59,286 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 752, max: 752
2015-02-10 22:48:59,287 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:59,288 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:59,288 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:59,288 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 55; bufvoid = 104857600
2015-02-10 22:48:59,288 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:48:59,291 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:59,293 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local725792397_0034_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:59,295 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/5847483701_source_77D070BA7154495FB0F0EE34ABC63BF8/part-00000:0+160
2015-02-10 22:48:59,296 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local725792397_0034_m_000000_0' done.
2015-02-10 22:48:59,296 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local725792397_0034_m_000000_0
2015-02-10 22:48:59,296 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:59,297 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:59,297 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local725792397_0034_r_000000_0
2015-02-10 22:48:59,299 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:59,299 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@6a905eb5
2015-02-10 22:48:59,300 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552658496, maxSingleShuffleLimit=138164624, mergeThreshold=364754624, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:59,301 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local725792397_0034_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:59,302 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#27 about to shuffle output of map attempt_local725792397_0034_m_000000_0 decomp: 67 len: 71 to MEMORY
2015-02-10 22:48:59,303 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 67 bytes from map-output for attempt_local725792397_0034_m_000000_0
2015-02-10 22:48:59,303 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 67, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->67
2015-02-10 22:48:59,304 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:59,305 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:59,307 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:59,309 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:59,309 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 22:48:59,310 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 67 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:59,311 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 71 bytes from disk
2015-02-10 22:48:59,311 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:59,311 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:59,312 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 22:48:59,313 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:59,313 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:59,313 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:59,319 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count-rhs)[by:[{1}:'char']]
2015-02-10 22:48:59,320 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][7584469547/count-rhs/]
2015-02-10 22:48:59,329 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local725792397_0034_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:59,331 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:59,331 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local725792397_0034_r_000000_0 is allowed to commit now
2015-02-10 22:48:59,332 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local725792397_0034_r_000000_0' to file:/tmp/hadoop-teamcity/7584469547_count_rhs_9A39F6A87410482384A9CB4D394AF522/_temporary/0/task_local725792397_0034_r_000000
2015-02-10 22:48:59,333 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:59,333 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local725792397_0034_r_000000_0' done.
2015-02-10 22:48:59,333 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local725792397_0034_r_000000_0
2015-02-10 22:48:59,333 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:59,349 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (2/5)
2015-02-10 22:48:59,350 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:59,351 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:59,393 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:59,406 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:59,427 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 22:48:59,437 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:59,437 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:59,443 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1447370509_0035
2015-02-10 22:48:59,462 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1447370509/.staging/job_local1447370509_0035/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:59,463 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1447370509/.staging/job_local1447370509_0035/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:59,517 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1447370509_0035/job_local1447370509_0035.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:59,518 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1447370509_0035/job_local1447370509_0035.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:59,519 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:59,520 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1447370509_0035
2015-02-10 22:48:59,520 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:59,520 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-10 22:48:59,521 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:59,523 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:59,523 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1447370509_0035_m_000000_0
2015-02-10 22:48:59,525 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:59,526 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/5847483701_source_77D070BA7154495FB0F0EE34ABC63BF8/part-00000
2015-02-10 22:48:59,527 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@289dc0a8
2015-02-10 22:48:59,528 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:59,528 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:59,558 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:59,558 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:59,558 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:59,558 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:59,558 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:59,561 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:59,561 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:59,567 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][5847483701/source/]
2015-02-10 22:48:59,567 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 22:48:59,569 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:59,570 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:59,570 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:59,570 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 65; bufvoid = 104857600
2015-02-10 22:48:59,570 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:48:59,573 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:59,574 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1447370509_0035_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:59,576 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/5847483701_source_77D070BA7154495FB0F0EE34ABC63BF8/part-00000:0+160
2015-02-10 22:48:59,576 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1447370509_0035_m_000000_0' done.
2015-02-10 22:48:59,576 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1447370509_0035_m_000000_0
2015-02-10 22:48:59,576 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1447370509_0035_m_000001_0
2015-02-10 22:48:59,578 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:59,579 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/7584469547_count_rhs_9A39F6A87410482384A9CB4D394AF522/part-00000
2015-02-10 22:48:59,579 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@612bd56c
2015-02-10 22:48:59,580 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:59,580 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:59,604 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:59,604 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:59,604 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:59,604 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:59,604 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:59,606 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:59,606 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:59,611 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][7584469547/count-rhs/]
2015-02-10 22:48:59,611 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 22:48:59,614 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:59,614 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:59,614 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:59,614 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 65; bufvoid = 104857600
2015-02-10 22:48:59,614 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:48:59,617 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:59,619 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1447370509_0035_m_000001_0 is done. And is in the process of committing
2015-02-10 22:48:59,620 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/7584469547_count_rhs_9A39F6A87410482384A9CB4D394AF522/part-00000:0+160
2015-02-10 22:48:59,620 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1447370509_0035_m_000001_0' done.
2015-02-10 22:48:59,621 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1447370509_0035_m_000001_0
2015-02-10 22:48:59,621 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:59,621 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:59,622 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1447370509_0035_r_000000_0
2015-02-10 22:48:59,623 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:59,623 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@674009d2
2015-02-10 22:48:59,624 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552153920, maxSingleShuffleLimit=138038480, mergeThreshold=364421600, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:59,624 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1447370509_0035_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:59,627 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#28 about to shuffle output of map attempt_local1447370509_0035_m_000000_0 decomp: 77 len: 81 to MEMORY
2015-02-10 22:48:59,627 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 77 bytes from map-output for attempt_local1447370509_0035_m_000000_0
2015-02-10 22:48:59,628 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 77, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->77
2015-02-10 22:48:59,630 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#28 about to shuffle output of map attempt_local1447370509_0035_m_000001_0 decomp: 77 len: 81 to MEMORY
2015-02-10 22:48:59,630 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 77 bytes from map-output for attempt_local1447370509_0035_m_000001_0
2015-02-10 22:48:59,630 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 77, inMemoryMapOutputs.size() -> 2, commitMemory -> 77, usedMemory ->154
2015-02-10 22:48:59,631 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:59,632 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:59,632 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:59,634 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 2 sorted segments
2015-02-10 22:48:59,634 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 2 segments left of total size: 134 bytes
2015-02-10 22:48:59,635 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 154 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:59,636 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 156 bytes from disk
2015-02-10 22:48:59,636 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:59,636 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:59,637 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 142 bytes
2015-02-10 22:48:59,637 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:59,638 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:59,638 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:59,643 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 22:48:59,643 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'num', 'char2', 'count']]"][4454370365/source_count-rhs/]
2015-02-10 22:48:59,658 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(105)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-10 22:48:59,658 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(110)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-10 22:48:59,665 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1447370509_0035_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:59,666 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:59,667 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1447370509_0035_r_000000_0 is allowed to commit now
2015-02-10 22:48:59,667 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1447370509_0035_r_000000_0' to file:/tmp/hadoop-teamcity/4454370365_source_count_rhs_FBF7F87AC9FE454E807704D45E0EB6C7/_temporary/0/task_local1447370509_0035_r_000000
2015-02-10 22:48:59,668 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:59,668 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1447370509_0035_r_000000_0' done.
2015-02-10 22:48:59,669 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1447370509_0035_r_000000_0
2015-02-10 22:48:59,669 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:59,685 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (5/5) ...m/samesourcemergethreeway
2015-02-10 22:48:59,686 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:59,687 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:59,729 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:59,743 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:59,763 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 22:48:59,773 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:59,773 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:59,779 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1541484361_0036
2015-02-10 22:48:59,806 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1541484361/.staging/job_local1541484361_0036/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:59,807 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1541484361/.staging/job_local1541484361_0036/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:59,855 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1541484361_0036/job_local1541484361_0036.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:48:59,857 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1541484361_0036/job_local1541484361_0036.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:48:59,857 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:59,857 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1541484361_0036
2015-02-10 22:48:59,858 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-10 22:48:59,858 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:59,858 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:59,862 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:59,862 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1541484361_0036_m_000000_0
2015-02-10 22:48:59,863 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:59,864 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/4454370365_source_count_rhs_FBF7F87AC9FE454E807704D45E0EB6C7/part-00000
2015-02-10 22:48:59,864 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@11e71d39
2015-02-10 22:48:59,866 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:59,866 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:59,890 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:59,890 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:59,890 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:59,890 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:59,890 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:59,892 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:59,893 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:59,900 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'num', 'char2', 'count']]"][4454370365/source_count-rhs/]
2015-02-10 22:48:59,900 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2015-02-10 22:48:59,902 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:59,902 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:59,903 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:59,903 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 105; bufvoid = 104857600
2015-02-10 22:48:59,903 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:48:59,905 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:59,907 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1541484361_0036_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:59,909 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/4454370365_source_count_rhs_FBF7F87AC9FE454E807704D45E0EB6C7/part-00000:0+200
2015-02-10 22:48:59,909 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1541484361_0036_m_000000_0' done.
2015-02-10 22:48:59,909 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1541484361_0036_m_000000_0
2015-02-10 22:48:59,909 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1541484361_0036_m_000001_0
2015-02-10 22:48:59,910 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:59,912 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/152035209_count_lhs_856A3A2E025F419F972FB242029AC240/part-00000
2015-02-10 22:48:59,912 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7ccf9a33
2015-02-10 22:48:59,913 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:59,914 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:59,937 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:59,937 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:59,938 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:59,938 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:59,938 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:59,939 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:59,939 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:59,946 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][152035209/count-lhs/]
2015-02-10 22:48:59,946 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2015-02-10 22:48:59,949 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:48:59,950 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:59,950 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:59,950 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 65; bufvoid = 104857600
2015-02-10 22:48:59,950 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:48:59,953 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:59,955 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1541484361_0036_m_000001_0 is done. And is in the process of committing
2015-02-10 22:48:59,956 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/152035209_count_lhs_856A3A2E025F419F972FB242029AC240/part-00000:0+160
2015-02-10 22:48:59,956 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1541484361_0036_m_000001_0' done.
2015-02-10 22:48:59,956 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1541484361_0036_m_000001_0
2015-02-10 22:48:59,957 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:59,957 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:59,958 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1541484361_0036_r_000000_0
2015-02-10 22:48:59,959 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:59,959 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5ee8730f
2015-02-10 22:48:59,960 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552199744, maxSingleShuffleLimit=138049936, mergeThreshold=364451840, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:59,961 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1541484361_0036_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:59,963 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#29 about to shuffle output of map attempt_local1541484361_0036_m_000000_0 decomp: 117 len: 121 to MEMORY
2015-02-10 22:48:59,964 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 117 bytes from map-output for attempt_local1541484361_0036_m_000000_0
2015-02-10 22:48:59,964 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 117, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->117
2015-02-10 22:48:59,967 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#29 about to shuffle output of map attempt_local1541484361_0036_m_000001_0 decomp: 77 len: 81 to MEMORY
2015-02-10 22:48:59,968 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 77 bytes from map-output for attempt_local1541484361_0036_m_000001_0
2015-02-10 22:48:59,968 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 77, inMemoryMapOutputs.size() -> 2, commitMemory -> 117, usedMemory ->194
2015-02-10 22:48:59,969 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:59,970 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:59,970 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:59,972 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 2 sorted segments
2015-02-10 22:48:59,973 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 2 segments left of total size: 174 bytes
2015-02-10 22:48:59,974 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 194 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:59,975 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 196 bytes from disk
2015-02-10 22:48:59,975 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:59,975 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:59,976 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 182 bytes
2015-02-10 22:48:59,977 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:59,977 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:59,977 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:59,984 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2015-02-10 22:48:59,984 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway"]
2015-02-10 22:48:59,992 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(105)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-10 22:48:59,992 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(110)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-10 22:48:59,995 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1541484361_0036_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:59,996 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:59,996 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1541484361_0036_r_000000_0 is allowed to commit now
2015-02-10 22:48:59,997 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1541484361_0036_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway/_temporary/0/task_local1541484361_0036_r_000000
2015-02-10 22:48:59,998 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:59,998 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1541484361_0036_r_000000_0' done.
2015-02-10 22:48:59,999 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1541484361_0036_r_000000_0
2015-02-10 22:48:59,999 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:49:00,019 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway/_temporary
2015-02-10 22:49:00,052 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:49:00,086 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:49:00,089 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway
2015-02-10 22:49:00,123 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:49:00,124 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:49:00,132 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:49:00,135 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:49:00,135 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:49:00,143 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs] starting
2015-02-10 22:49:00,144 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:49:00,144 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:49:00,144 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs]  sink: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic"]
2015-02-10 22:49:00,144 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs]  parallel execution is enabled: false
2015-02-10 22:49:00,145 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs]  starting jobs: 1
2015-02-10 22:49:00,145 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs]  allocating threads: 1
2015-02-10 22:49:00,146 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lhs+rhs] starting step: (1/1) ...quemerge-nondeterministic
2015-02-10 22:49:00,147 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:49:00,148 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:49:00,173 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:49:00,174 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:49:00,192 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 22:49:00,208 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1200652964_0037
2015-02-10 22:49:00,226 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1200652964/.staging/job_local1200652964_0037/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:49:00,228 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1200652964/.staging/job_local1200652964_0037/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:49:00,273 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1200652964_0037/job_local1200652964_0037.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:49:00,274 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1200652964_0037/job_local1200652964_0037.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:49:00,275 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:49:00,275 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lhs+rhs] submitted hadoop job: job_local1200652964_0037
2015-02-10 22:49:00,275 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:49:00,275 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lhs+rhs] tracking url: http://localhost:8080/
2015-02-10 22:49:00,276 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:49:00,278 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:49:00,279 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1200652964_0037_m_000000_0
2015-02-10 22:49:00,280 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:49:00,281 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:49:00,281 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@43798b1a
2015-02-10 22:49:00,282 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:49:00,283 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:49:00,304 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:49:00,304 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:49:00,304 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:49:00,304 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:49:00,304 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:49:00,306 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:49:00,306 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:49:00,310 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:49:00,311 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2015-02-10 22:49:00,317 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:49:00,317 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:49:00,317 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:49:00,317 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-10 22:49:00,317 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:49:00,320 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:49:00,321 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1200652964_0037_m_000000_0 is done. And is in the process of committing
2015-02-10 22:49:00,323 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:49:00,323 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1200652964_0037_m_000000_0' done.
2015-02-10 22:49:00,323 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1200652964_0037_m_000000_0
2015-02-10 22:49:00,324 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1200652964_0037_m_000001_0
2015-02-10 22:49:00,325 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:49:00,326 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt
2015-02-10 22:49:00,326 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@df61ca
2015-02-10 22:49:00,327 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:49:00,327 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:49:00,358 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:49:00,358 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:49:00,358 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:49:00,358 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:49:00,358 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:49:00,360 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:49:00,360 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:49:00,364 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:49:00,364 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2015-02-10 22:49:00,368 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:49:00,368 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:49:00,368 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:49:00,368 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-10 22:49:00,368 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:49:00,371 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:49:00,372 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1200652964_0037_m_000001_0 is done. And is in the process of committing
2015-02-10 22:49:00,374 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 22:49:00,374 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1200652964_0037_m_000001_0' done.
2015-02-10 22:49:00,374 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1200652964_0037_m_000001_0
2015-02-10 22:49:00,374 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:49:00,375 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:49:00,375 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1200652964_0037_r_000000_0
2015-02-10 22:49:00,376 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:49:00,377 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@6e7c6928
2015-02-10 22:49:00,377 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552429120, maxSingleShuffleLimit=138107280, mergeThreshold=364603232, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:49:00,379 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1200652964_0037_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:49:00,383 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#30 about to shuffle output of map attempt_local1200652964_0037_m_000000_0 decomp: 87 len: 91 to MEMORY
2015-02-10 22:49:00,384 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local1200652964_0037_m_000000_0
2015-02-10 22:49:00,384 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2015-02-10 22:49:00,386 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#30 about to shuffle output of map attempt_local1200652964_0037_m_000001_0 decomp: 87 len: 91 to MEMORY
2015-02-10 22:49:00,386 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local1200652964_0037_m_000001_0
2015-02-10 22:49:00,386 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 2, commitMemory -> 87, usedMemory ->174
2015-02-10 22:49:00,387 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:49:00,388 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:49:00,389 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:49:00,391 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 2 sorted segments
2015-02-10 22:49:00,391 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 2 segments left of total size: 156 bytes
2015-02-10 22:49:00,392 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 174 bytes to disk to satisfy reduce memory limit
2015-02-10 22:49:00,393 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 176 bytes from disk
2015-02-10 22:49:00,393 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:49:00,393 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:49:00,393 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 163 bytes
2015-02-10 22:49:00,394 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:49:00,394 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:49:00,394 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:49:00,399 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2015-02-10 22:49:00,399 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic"]
2015-02-10 22:49:00,413 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1200652964_0037_r_000000_0 is done. And is in the process of committing
2015-02-10 22:49:00,414 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:49:00,414 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1200652964_0037_r_000000_0 is allowed to commit now
2015-02-10 22:49:00,415 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1200652964_0037_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic/_temporary/0/task_local1200652964_0037_r_000000
2015-02-10 22:49:00,416 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:49:00,416 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1200652964_0037_r_000000_0' done.
2015-02-10 22:49:00,416 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1200652964_0037_r_000000_0
2015-02-10 22:49:00,417 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:49:00,439 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic/_temporary
2015-02-10 22:49:00,468 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:49:00,470 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic
2015-02-10 22:49:00,490 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:49:00,490 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:49:00,496 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:49:00,498 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:49:00,499 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:49:00,503 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 22:49:00,503 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:49:00,504 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  sink: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique"]
2015-02-10 22:49:00,504 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  parallel execution is enabled: false
2015-02-10 22:49:00,505 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  starting jobs: 1
2015-02-10 22:49:00,505 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  allocating threads: 1
2015-02-10 22:49:00,506 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...blyhelpersplatform/unique
2015-02-10 22:49:00,507 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:49:00,508 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:49:00,533 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:49:00,554 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:49:00,570 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1535422660_0038
2015-02-10 22:49:00,586 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1535422660/.staging/job_local1535422660_0038/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:49:00,587 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1535422660/.staging/job_local1535422660_0038/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:49:00,637 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1535422660_0038/job_local1535422660_0038.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:49:00,638 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1535422660_0038/job_local1535422660_0038.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:49:00,638 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:49:00,639 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1535422660_0038
2015-02-10 22:49:00,639 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 22:49:00,639 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:49:00,640 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:49:00,643 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:49:00,643 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1535422660_0038_m_000000_0
2015-02-10 22:49:00,644 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:49:00,645 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:49:00,646 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5295c398
2015-02-10 22:49:00,647 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:49:00,647 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:49:00,682 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:49:00,682 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:49:00,682 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:49:00,682 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:49:00,682 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:49:00,684 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:49:00,684 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:49:00,688 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:49:00,688 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(shape)[by:[{1}:'num']]
2015-02-10 22:49:00,692 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:49:00,692 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:49:00,692 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:49:00,692 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-10 22:49:00,692 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:49:00,695 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:49:00,697 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1535422660_0038_m_000000_0 is done. And is in the process of committing
2015-02-10 22:49:00,699 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:49:00,699 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1535422660_0038_m_000000_0' done.
2015-02-10 22:49:00,699 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1535422660_0038_m_000000_0
2015-02-10 22:49:00,699 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:49:00,700 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:49:00,700 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1535422660_0038_r_000000_0
2015-02-10 22:49:00,701 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:49:00,701 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@70afe432
2015-02-10 22:49:00,702 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=550548288, maxSingleShuffleLimit=137637072, mergeThreshold=363361888, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:49:00,703 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1535422660_0038_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:49:00,706 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#31 about to shuffle output of map attempt_local1535422660_0038_m_000000_0 decomp: 87 len: 91 to MEMORY
2015-02-10 22:49:00,707 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local1535422660_0038_m_000000_0
2015-02-10 22:49:00,707 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2015-02-10 22:49:00,707 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:49:00,708 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:49:00,709 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:49:00,710 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:49:00,710 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 22:49:00,712 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 87 bytes to disk to satisfy reduce memory limit
2015-02-10 22:49:00,712 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 91 bytes from disk
2015-02-10 22:49:00,712 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:49:00,712 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:49:00,713 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 22:49:00,713 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:49:00,713 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:49:00,714 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:49:00,717 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(shape)[by:[{1}:'num']]
2015-02-10 22:49:00,717 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique"]
2015-02-10 22:49:00,727 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1535422660_0038_r_000000_0 is done. And is in the process of committing
2015-02-10 22:49:00,728 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:49:00,728 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1535422660_0038_r_000000_0 is allowed to commit now
2015-02-10 22:49:00,729 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1535422660_0038_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique/_temporary/0/task_local1535422660_0038_r_000000
2015-02-10 22:49:00,730 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:49:00,730 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1535422660_0038_r_000000_0' done.
2015-02-10 22:49:00,730 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1535422660_0038_r_000000_0
2015-02-10 22:49:00,731 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:49:00,746 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique/_temporary
2015-02-10 22:49:00,779 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:49:00,781 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/unique
2015-02-10 22:49:00,806 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:49:00,806 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:49:00,813 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:49:00,815 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:49:00,816 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:49:00,821 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting
2015-02-10 22:49:00,821 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:49:00,821 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  sink: Hfs["TextDelimited[['notnull', 'null']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull"]
2015-02-10 22:49:00,822 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  parallel execution is enabled: false
2015-02-10 22:49:00,822 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  starting jobs: 1
2015-02-10 22:49:00,822 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  allocating threads: 1
2015-02-10 22:49:00,823 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...platform/countnullnotnull
2015-02-10 22:49:00,824 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:49:00,825 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:49:00,848 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:49:00,868 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:49:00,884 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local929874522_0039
2015-02-10 22:49:00,901 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity929874522/.staging/job_local929874522_0039/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:49:00,902 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity929874522/.staging/job_local929874522_0039/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:49:00,949 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local929874522_0039/job_local929874522_0039.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 22:49:00,951 WARN  conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local929874522_0039/job_local929874522_0039.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 22:49:00,951 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:49:00,952 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local929874522_0039
2015-02-10 22:49:00,952 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:49:00,952 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-10 22:49:00,952 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:49:00,955 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:49:00,955 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local929874522_0039_m_000000_0
2015-02-10 22:49:00,957 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:49:00,959 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:49:00,959 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@37ba658a
2015-02-10 22:49:00,960 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:49:00,961 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:49:00,990 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:49:00,991 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:49:00,991 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:49:00,991 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:49:00,991 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:49:00,992 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:49:00,992 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:49:00,999 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:49:01,000 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{?}:NONE]]
2015-02-10 22:49:01,000 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:49:01,007 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 22:49:01,008 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:49:01,008 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:49:01,008 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 6; bufvoid = 104857600
2015-02-10 22:49:01,008 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214396(104857584); length = 1/6553600
2015-02-10 22:49:01,010 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:49:01,012 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local929874522_0039_m_000000_0 is done. And is in the process of committing
2015-02-10 22:49:01,014 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:49:01,014 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local929874522_0039_m_000000_0' done.
2015-02-10 22:49:01,014 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local929874522_0039_m_000000_0
2015-02-10 22:49:01,014 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:49:01,015 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:49:01,015 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local929874522_0039_r_000000_0
2015-02-10 22:49:01,017 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:49:01,017 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@33319df3
2015-02-10 22:49:01,017 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552337408, maxSingleShuffleLimit=138084352, mergeThreshold=364542688, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:49:01,018 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local929874522_0039_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:49:01,019 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#32 about to shuffle output of map attempt_local929874522_0039_m_000000_0 decomp: 10 len: 14 to MEMORY
2015-02-10 22:49:01,020 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 10 bytes from map-output for attempt_local929874522_0039_m_000000_0
2015-02-10 22:49:01,020 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 10, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->10
2015-02-10 22:49:01,021 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:49:01,022 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:49:01,022 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:49:01,024 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:49:01,025 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 7 bytes
2015-02-10 22:49:01,026 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 10 bytes to disk to satisfy reduce memory limit
2015-02-10 22:49:01,026 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 14 bytes from disk
2015-02-10 22:49:01,026 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:49:01,026 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:49:01,027 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 7 bytes
2015-02-10 22:49:01,027 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:49:01,028 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:49:01,028 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:49:01,034 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{?}:NONE]]
2015-02-10 22:49:01,034 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['notnull', 'null']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull"]
2015-02-10 22:49:01,044 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local929874522_0039_r_000000_0 is done. And is in the process of committing
2015-02-10 22:49:01,045 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:49:01,045 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local929874522_0039_r_000000_0 is allowed to commit now
2015-02-10 22:49:01,046 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local929874522_0039_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull/_temporary/0/task_local929874522_0039_r_000000
2015-02-10 22:49:01,047 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:49:01,047 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local929874522_0039_r_000000_0' done.
2015-02-10 22:49:01,047 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local929874522_0039_r_000000_0
2015-02-10 22:49:01,047 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:49:01,066 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull/_temporary
2015-02-10 22:49:01,096 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:49:01,125 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:49:01,127 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/countnullnotnull