Class cascading.pipe.assembly.AssemblyHelpersPlatformTest

all > cascading.pipe.assembly > AssemblyHelpersPlatformTest
tests
failures
ignored
8.199s
duration
100%
successful
Tests
Standard output
Tests

Test	Duration	Result
testAverageBy	0.179s	passed
testAverageByNull	0.181s	passed
testAverageMerge	0.292s	passed
testCoerce	0.140s	passed
testCoerceFields	0.173s	passed
testCount	0.218s	passed
testCountAll	0.192s	passed
testCountCount	0.461s	passed
testCountMerge	0.285s	passed
testCountNullNotNull	0.204s	passed
testDiscardNarrow	0.152s	passed
testFirstBy	0.156s	passed
testMaxBy	0.253s	passed
testMaxByString	0.197s	passed
testMinBy	0.218s	passed
testMinByString	0.279s	passed
testParallelAggregates	0.181s	passed
testParallelAggregatesMerge	1.472s	passed
testParallelAggregatesPriorMerge	0.269s	passed
testRenameAll	0.171s	passed
testRenameNamed	0.137s	passed
testRenameNarrow	0.118s	passed
testRetainNarrow	0.121s	passed
testSameSourceMerge	0.316s	passed
testSameSourceMergeThreeWay	0.798s	passed
testSumBy	0.222s	passed
testSumByNulls	0.197s	passed
testSumMerge	0.254s	passed
testUnique	0.162s	passed
testUniqueMerge	0.201s	passed
Standard output

2014-04-10 11:46:27,574 INFO  platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(187)) - found 1 test platforms from classpath
2014-04-10 11:46:27,598 INFO  platform.PlatformRunner (PlatformRunner.java:addPlatform(237)) - installing platform: hadoop2-mr1
2014-04-10 11:46:27,599 INFO  platform.PlatformRunner (PlatformRunner.java:addPlatform(238)) - running test: cascading.pipe.assembly.AssemblyHelpersPlatformTest
2014-04-10 11:46:27,614 INFO  hadoop2.Hadoop2MR1Platform (Hadoop2MR1Platform.java:setUp(106)) - not using cluster
2014-04-10 11:46:27,890 WARN  util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2014-04-10 11:46:28,150 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:28,151 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:28,157 INFO  property.AppProps (AppProps.java:getAppID(162)) - using app.id: 05B1F38C5CED440894C793910CB222FF
2014-04-10 11:46:28,201 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:28,218 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:28,220 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:28,281 INFO  util.Version (Version.java:printBanner(78)) - Concurrent, Inc - Cascading 2.5.3
2014-04-10 11:46:28,284 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [name] starting
2014-04-10 11:46:28,284 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [name]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt"]
2014-04-10 11:46:28,285 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [name]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:28,285 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [name]  sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false"]
2014-04-10 11:46:28,287 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [name]  parallel execution is enabled: false
2014-04-10 11:46:28,287 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [name]  starting jobs: 1
2014-04-10 11:46:28,288 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [name]  allocating threads: 1
2014-04-10 11:46:28,289 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] starting step: (1/1) ...platform/multimerge+false
2014-04-10 11:46:28,305 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - session.id is deprecated. Instead, use dfs.metrics.session-id
2014-04-10 11:46:28,305 INFO  jvm.JvmMetrics (JvmMetrics.java:init(76)) - Initializing JVM Metrics with processName=JobTracker, sessionId=
2014-04-10 11:46:28,315 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:28,423 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:28,427 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:28,441 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2014-04-10 11:46:28,550 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local856396545_0001
2014-04-10 11:46:28,581 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock856396545/.staging/job_local856396545_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:28,581 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock856396545/.staging/job_local856396545_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:28,670 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local856396545_0001/job_local856396545_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:28,671 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local856396545_0001/job_local856396545_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:28,677 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:28,677 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:28,678 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] submitted hadoop job: job_local856396545_0001
2014-04-10 11:46:28,678 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] tracking url: http://localhost:8080/
2014-04-10 11:46:28,678 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:28,705 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:28,707 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local856396545_0001_m_000000_0
2014-04-10 11:46:28,723 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:28,731 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:28,731 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3ad4bd11
2014-04-10 11:46:28,735 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:28,738 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:28,749 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:28,749 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:28,749 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:28,749 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:28,749 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:28,759 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:28,759 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:28,786 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
2014-04-10 11:46:28,789 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:28,789 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2014-04-10 11:46:28,793 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:28,801 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:28,801 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 328, total: 480, max: 672
2014-04-10 11:46:28,801 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 71% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:28,802 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:28,802 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 328, total: 480, max: 672
2014-04-10 11:46:28,802 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 71% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:28,803 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-04-10 11:46:28,803 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 328, total: 480, max: 672
2014-04-10 11:46:28,803 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 71% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:28,803 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-04-10 11:46:28,803 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 328, total: 480, max: 672
2014-04-10 11:46:28,803 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 71% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:28,805 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:28,805 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:28,805 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:28,805 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2014-04-10 11:46:28,806 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2014-04-10 11:46:28,813 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:28,815 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local856396545_0001_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:28,819 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:28,819 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local856396545_0001_m_000000_0' done.
2014-04-10 11:46:28,820 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local856396545_0001_m_000000_0
2014-04-10 11:46:28,820 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local856396545_0001_m_000001_0
2014-04-10 11:46:28,821 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:28,822 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt
2014-04-10 11:46:28,822 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5473f8ac
2014-04-10 11:46:28,823 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:28,823 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:28,833 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:28,833 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:28,833 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:28,833 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:28,833 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:28,834 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:28,835 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:28,846 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt"]
2014-04-10 11:46:28,846 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2014-04-10 11:46:28,847 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:28,894 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:28,894 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 223, total: 480, max: 672
2014-04-10 11:46:28,894 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 71% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:28,895 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:28,895 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 223, total: 480, max: 672
2014-04-10 11:46:28,895 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 71% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:28,896 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-04-10 11:46:28,896 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 223, total: 480, max: 672
2014-04-10 11:46:28,896 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 71% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:28,896 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-04-10 11:46:28,896 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 223, total: 480, max: 672
2014-04-10 11:46:28,896 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 71% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:28,897 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:28,897 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:28,897 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:28,897 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2014-04-10 11:46:28,897 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2014-04-10 11:46:28,900 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:28,901 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local856396545_0001_m_000001_0 is done. And is in the process of committing
2014-04-10 11:46:28,902 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt:0+51
2014-04-10 11:46:28,902 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local856396545_0001_m_000001_0' done.
2014-04-10 11:46:28,902 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local856396545_0001_m_000001_0
2014-04-10 11:46:28,903 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:28,904 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:28,904 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local856396545_0001_r_000000_0
2014-04-10 11:46:28,908 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:28,909 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@60c3e273
2014-04-10 11:46:28,915 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:28,917 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local856396545_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:28,935 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#1 about to shuffle output of map attempt_local856396545_0001_m_000000_0 decomp: 236 len: 240 to MEMORY
2014-04-10 11:46:28,937 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local856396545_0001_m_000000_0
2014-04-10 11:46:28,953 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->236
2014-04-10 11:46:28,954 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#1 about to shuffle output of map attempt_local856396545_0001_m_000001_0 decomp: 236 len: 240 to MEMORY
2014-04-10 11:46:28,955 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local856396545_0001_m_000001_0
2014-04-10 11:46:28,955 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 2, commitMemory -> 236, usedMemory ->472
2014-04-10 11:46:28,955 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:28,956 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:28,956 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:28,961 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2014-04-10 11:46:28,962 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 454 bytes
2014-04-10 11:46:28,963 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 472 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:28,963 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 474 bytes from disk
2014-04-10 11:46:28,964 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:28,964 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:28,964 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 461 bytes
2014-04-10 11:46:28,964 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:28,965 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:28,965 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:28,977 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2014-04-10 11:46:28,978 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false"]
2014-04-10 11:46:28,991 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local856396545_0001_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:28,992 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:28,992 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local856396545_0001_r_000000_0 is allowed to commit now
2014-04-10 11:46:28,993 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local856396545_0001_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false/_temporary/0/task_local856396545_0001_r_000000
2014-04-10 11:46:28,993 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:28,994 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local856396545_0001_r_000000_0' done.
2014-04-10 11:46:28,994 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local856396545_0001_r_000000_0
2014-04-10 11:46:28,994 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:29,028 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false/_temporary
2014-04-10 11:46:29,055 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:29,081 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:29,084 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/multimerge+false
2014-04-10 11:46:29,107 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:29,108 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:29,116 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:29,119 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:29,119 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:29,130 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [max] starting
2014-04-10 11:46:29,130 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [max]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:29,130 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [max]  sink: Hfs["TextDelimited[['num', 'min']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring"]
2014-04-10 11:46:29,130 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [max]  parallel execution is enabled: false
2014-04-10 11:46:29,131 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [max]  starting jobs: 1
2014-04-10 11:46:29,131 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [max]  allocating threads: 1
2014-04-10 11:46:29,131 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...lpersplatform/minbystring
2014-04-10 11:46:29,132 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:29,133 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:29,157 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:29,168 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:29,186 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local552050982_0002
2014-04-10 11:46:29,196 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock552050982/.staging/job_local552050982_0002/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:29,196 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock552050982/.staging/job_local552050982_0002/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:29,237 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local552050982_0002/job_local552050982_0002.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:29,238 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local552050982_0002/job_local552050982_0002.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:29,238 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:29,239 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local552050982_0002
2014-04-10 11:46:29,239 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:29,239 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2014-04-10 11:46:29,239 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:29,241 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:29,241 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local552050982_0002_m_000000_0
2014-04-10 11:46:29,242 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:29,243 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:29,243 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3fa123af
2014-04-10 11:46:29,245 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:29,245 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:29,255 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:29,255 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:29,255 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:29,255 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:29,255 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:29,256 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:29,256 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:29,262 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:29,262 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(max)[by:[{1}:'num']]
2014-04-10 11:46:29,263 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:29,264 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:29,264 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 59, total: 480, max: 672
2014-04-10 11:46:29,264 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 71% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:29,264 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:29,264 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 59, total: 480, max: 672
2014-04-10 11:46:29,264 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 71% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:29,265 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:29,265 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:29,265 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:29,265 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2014-04-10 11:46:29,266 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2014-04-10 11:46:29,267 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:29,269 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local552050982_0002_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:29,271 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:29,271 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local552050982_0002_m_000000_0' done.
2014-04-10 11:46:29,271 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local552050982_0002_m_000000_0
2014-04-10 11:46:29,271 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:29,272 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:29,272 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local552050982_0002_r_000000_0
2014-04-10 11:46:29,274 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:29,275 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5d8eee3b
2014-04-10 11:46:29,275 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:29,276 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local552050982_0002_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:29,277 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#2 about to shuffle output of map attempt_local552050982_0002_m_000000_0 decomp: 87 len: 91 to MEMORY
2014-04-10 11:46:29,277 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local552050982_0002_m_000000_0
2014-04-10 11:46:29,277 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2014-04-10 11:46:29,278 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:29,278 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:29,279 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:29,282 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:29,283 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2014-04-10 11:46:29,285 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 87 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:29,286 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 91 bytes from disk
2014-04-10 11:46:29,286 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:29,287 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:29,287 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2014-04-10 11:46:29,288 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:29,288 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:29,289 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:29,296 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(max)[by:[{1}:'num']]
2014-04-10 11:46:29,296 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['num', 'min']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring"]
2014-04-10 11:46:29,304 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local552050982_0002_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:29,305 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:29,306 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local552050982_0002_r_000000_0 is allowed to commit now
2014-04-10 11:46:29,307 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local552050982_0002_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring/_temporary/0/task_local552050982_0002_r_000000
2014-04-10 11:46:29,308 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:29,308 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local552050982_0002_r_000000_0' done.
2014-04-10 11:46:29,308 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local552050982_0002_r_000000_0
2014-04-10 11:46:29,308 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:29,329 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring/_temporary
2014-04-10 11:46:29,348 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:29,364 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:29,366 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/minbystring
2014-04-10 11:46:29,382 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:29,382 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:29,403 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:29,409 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:29,409 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:29,418 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [average-lhs+average-rhs] starting
2014-04-10 11:46:29,418 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [average-lhs+average-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt"]
2014-04-10 11:46:29,418 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [average-lhs+average-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:29,418 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [average-lhs+average-rhs]  sink: Hfs["TextDelimited[['char', 'average']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage"]
2014-04-10 11:46:29,418 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [average-lhs+average-rhs]  parallel execution is enabled: false
2014-04-10 11:46:29,418 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [average-lhs+average-rhs]  starting jobs: 1
2014-04-10 11:46:29,418 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [average-lhs+average-rhs]  allocating threads: 1
2014-04-10 11:46:29,419 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average-lhs+average-rhs] starting step: (1/1) ...persplatform/mergeaverage
2014-04-10 11:46:29,419 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:29,420 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:29,432 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:29,433 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:29,440 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2014-04-10 11:46:29,457 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1492738268_0003
2014-04-10 11:46:29,470 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1492738268/.staging/job_local1492738268_0003/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:29,470 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1492738268/.staging/job_local1492738268_0003/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:29,502 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1492738268_0003/job_local1492738268_0003.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:29,502 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1492738268_0003/job_local1492738268_0003.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:29,503 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:29,503 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average-lhs+average-rhs] submitted hadoop job: job_local1492738268_0003
2014-04-10 11:46:29,503 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:29,504 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average-lhs+average-rhs] tracking url: http://localhost:8080/
2014-04-10 11:46:29,504 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:29,505 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:29,506 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1492738268_0003_m_000000_0
2014-04-10 11:46:29,507 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:29,509 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:29,509 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@536ae6f3
2014-04-10 11:46:29,510 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:29,510 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:29,521 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:29,521 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:29,521 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:29,521 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:29,521 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:29,522 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:29,522 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:29,530 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:29,530 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2014-04-10 11:46:29,531 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:29,531 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:29,531 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 137, total: 580, max: 672
2014-04-10 11:46:29,532 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:29,532 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 137, total: 580, max: 672
2014-04-10 11:46:29,533 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-04-10 11:46:29,533 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 137, total: 580, max: 672
2014-04-10 11:46:29,533 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-04-10 11:46:29,533 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 137, total: 580, max: 672
2014-04-10 11:46:29,534 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:29,534 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:29,534 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:29,535 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2014-04-10 11:46:29,535 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2014-04-10 11:46:29,537 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:29,538 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1492738268_0003_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:29,539 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:29,540 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1492738268_0003_m_000000_0' done.
2014-04-10 11:46:29,540 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1492738268_0003_m_000000_0
2014-04-10 11:46:29,540 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1492738268_0003_m_000001_0
2014-04-10 11:46:29,541 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:29,542 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt
2014-04-10 11:46:29,542 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7f3b2f29
2014-04-10 11:46:29,543 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:29,544 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:29,555 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:29,555 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:29,555 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:29,555 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:29,555 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:29,556 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:29,556 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:29,563 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt"]
2014-04-10 11:46:29,564 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2014-04-10 11:46:29,564 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:29,566 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:29,566 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 105, total: 650, max: 672
2014-04-10 11:46:29,567 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:29,567 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 105, total: 650, max: 672
2014-04-10 11:46:29,567 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-04-10 11:46:29,567 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 105, total: 650, max: 672
2014-04-10 11:46:29,568 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-04-10 11:46:29,568 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 105, total: 650, max: 672
2014-04-10 11:46:29,569 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:29,569 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:29,569 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:29,569 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2014-04-10 11:46:29,569 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2014-04-10 11:46:29,571 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:29,572 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1492738268_0003_m_000001_0 is done. And is in the process of committing
2014-04-10 11:46:29,573 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt:0+51
2014-04-10 11:46:29,573 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1492738268_0003_m_000001_0' done.
2014-04-10 11:46:29,573 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1492738268_0003_m_000001_0
2014-04-10 11:46:29,573 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:29,574 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:29,574 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1492738268_0003_r_000000_0
2014-04-10 11:46:29,575 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:29,575 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@10dde075
2014-04-10 11:46:29,576 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:29,576 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1492738268_0003_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:29,578 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#3 about to shuffle output of map attempt_local1492738268_0003_m_000000_0 decomp: 200 len: 204 to MEMORY
2014-04-10 11:46:29,578 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local1492738268_0003_m_000000_0
2014-04-10 11:46:29,578 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->200
2014-04-10 11:46:29,580 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#3 about to shuffle output of map attempt_local1492738268_0003_m_000001_0 decomp: 200 len: 204 to MEMORY
2014-04-10 11:46:29,580 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local1492738268_0003_m_000001_0
2014-04-10 11:46:29,580 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 2, commitMemory -> 200, usedMemory ->400
2014-04-10 11:46:29,581 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:29,581 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:29,581 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:29,583 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2014-04-10 11:46:29,583 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 382 bytes
2014-04-10 11:46:29,584 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 400 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:29,584 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 402 bytes from disk
2014-04-10 11:46:29,584 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:29,584 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:29,585 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 389 bytes
2014-04-10 11:46:29,585 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:29,585 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:29,585 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:29,592 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2014-04-10 11:46:29,592 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage"]
2014-04-10 11:46:29,598 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1492738268_0003_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:29,599 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:29,600 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1492738268_0003_r_000000_0 is allowed to commit now
2014-04-10 11:46:29,600 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1492738268_0003_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage/_temporary/0/task_local1492738268_0003_r_000000
2014-04-10 11:46:29,601 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:29,601 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1492738268_0003_r_000000_0' done.
2014-04-10 11:46:29,601 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1492738268_0003_r_000000_0
2014-04-10 11:46:29,601 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:29,621 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage/_temporary
2014-04-10 11:46:29,637 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:29,656 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:29,658 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/mergeaverage
2014-04-10 11:46:29,671 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:29,672 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:29,677 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:29,679 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:29,679 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:29,685 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] starting
2014-04-10 11:46:29,685 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt"]
2014-04-10 11:46:29,686 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  sink: Hfs["TextLine[['line']->['item', 'element']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall"]
2014-04-10 11:46:29,686 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  parallel execution is enabled: false
2014-04-10 11:46:29,686 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  starting jobs: 1
2014-04-10 11:46:29,686 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  allocating threads: 1
2014-04-10 11:46:29,686 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...helpersplatform/renameall
2014-04-10 11:46:29,686 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:29,687 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:29,697 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:29,704 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:29,718 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local2075282057_0004
2014-04-10 11:46:29,727 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock2075282057/.staging/job_local2075282057_0004/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:29,727 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock2075282057/.staging/job_local2075282057_0004/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:29,759 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local2075282057_0004/job_local2075282057_0004.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:29,760 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local2075282057_0004/job_local2075282057_0004.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:29,760 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:29,760 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:29,760 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local2075282057_0004
2014-04-10 11:46:29,760 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:29,761 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2014-04-10 11:46:29,763 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:29,763 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local2075282057_0004_m_000000_0
2014-04-10 11:46:29,766 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:29,767 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt
2014-04-10 11:46:29,768 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@30d271e7
2014-04-10 11:46:29,769 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2014-04-10 11:46:29,776 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:29,776 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:29,782 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt"]
2014-04-10 11:46:29,783 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: Hfs["TextLine[['line']->['item', 'element']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall"]
2014-04-10 11:46:29,784 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:29,784 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local2075282057_0004_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:29,785 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:29,785 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local2075282057_0004_m_000000_0 is allowed to commit now
2014-04-10 11:46:29,786 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local2075282057_0004_m_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall/_temporary/0/task_local2075282057_0004_m_000000
2014-04-10 11:46:29,786 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt:0+19
2014-04-10 11:46:29,787 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2075282057_0004_m_000000_0' done.
2014-04-10 11:46:29,787 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local2075282057_0004_m_000000_0
2014-04-10 11:46:29,787 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:29,807 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall/_temporary
2014-04-10 11:46:29,828 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:29,830 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/renameall
2014-04-10 11:46:29,846 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:29,846 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:29,856 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:29,857 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:29,865 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:29,870 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count] starting
2014-04-10 11:46:29,870 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:29,871 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count]  sink: Hfs["TextDelimited[['count', 'count2']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount"]
2014-04-10 11:46:29,871 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count]  parallel execution is enabled: false
2014-04-10 11:46:29,871 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count]  starting jobs: 2
2014-04-10 11:46:29,872 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count]  allocating threads: 1
2014-04-10 11:46:29,872 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/2)
2014-04-10 11:46:29,873 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:29,874 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:29,891 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:29,897 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:29,908 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local68332549_0005
2014-04-10 11:46:29,917 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock68332549/.staging/job_local68332549_0005/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:29,917 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock68332549/.staging/job_local68332549_0005/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:29,987 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local68332549_0005/job_local68332549_0005.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:29,987 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local68332549_0005/job_local68332549_0005.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:29,988 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:29,988 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:29,988 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local68332549_0005
2014-04-10 11:46:29,988 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:29,988 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2014-04-10 11:46:29,990 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:29,990 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local68332549_0005_m_000000_0
2014-04-10 11:46:29,993 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:29,994 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:29,994 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1a1d8eb0
2014-04-10 11:46:29,996 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:29,996 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:30,008 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:30,008 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:30,008 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:30,008 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:30,008 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:30,009 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:30,009 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:30,016 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:30,016 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(count)[by:[{1}:'char']]
2014-04-10 11:46:30,017 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:30,018 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:30,018 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 258, total: 372, max: 672
2014-04-10 11:46:30,018 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 55% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:30,018 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:30,018 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 258, total: 372, max: 672
2014-04-10 11:46:30,018 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 55% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:30,019 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-04-10 11:46:30,019 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 258, total: 372, max: 672
2014-04-10 11:46:30,019 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 55% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:30,019 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-04-10 11:46:30,019 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 258, total: 372, max: 672
2014-04-10 11:46:30,019 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 55% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:30,020 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:30,020 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:30,020 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:30,020 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2014-04-10 11:46:30,020 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2014-04-10 11:46:30,022 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:30,023 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local68332549_0005_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:30,024 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:30,024 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local68332549_0005_m_000000_0' done.
2014-04-10 11:46:30,024 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local68332549_0005_m_000000_0
2014-04-10 11:46:30,024 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:30,024 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:30,025 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local68332549_0005_r_000000_0
2014-04-10 11:46:30,026 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:30,026 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5650216a
2014-04-10 11:46:30,026 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:30,026 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local68332549_0005_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:30,028 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#4 about to shuffle output of map attempt_local68332549_0005_m_000000_0 decomp: 119 len: 123 to MEMORY
2014-04-10 11:46:30,028 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local68332549_0005_m_000000_0
2014-04-10 11:46:30,028 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2014-04-10 11:46:30,029 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:30,029 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:30,029 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:30,031 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:30,031 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2014-04-10 11:46:30,032 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:30,032 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2014-04-10 11:46:30,032 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:30,032 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:30,032 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2014-04-10 11:46:30,033 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:30,033 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:30,033 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:30,039 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(count)[by:[{1}:'char']]
2014-04-10 11:46:30,039 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: TempHfs["SequenceFile[['count', 'count2']]"][8506900574/count/]
2014-04-10 11:46:30,049 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:30,052 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:30,052 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 253, total: 372, max: 672
2014-04-10 11:46:30,053 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 55% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:30,053 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local68332549_0005_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:30,056 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:30,056 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local68332549_0005_r_000000_0 is allowed to commit now
2014-04-10 11:46:30,057 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local68332549_0005_r_000000_0' to file:/tmp/hadoop-brock/8506900574_count_412DBAF5170A4C4789F5E0FD5DC2386A/_temporary/0/task_local68332549_0005_r_000000
2014-04-10 11:46:30,058 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:30,058 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local68332549_0005_r_000000_0' done.
2014-04-10 11:46:30,058 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local68332549_0005_r_000000_0
2014-04-10 11:46:30,058 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:30,078 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (2/2) ...elpersplatform/countcount
2014-04-10 11:46:30,078 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:30,079 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:30,099 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:30,112 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:30,119 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:30,120 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:30,128 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local204610770_0006
2014-04-10 11:46:30,143 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock204610770/.staging/job_local204610770_0006/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:30,143 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock204610770/.staging/job_local204610770_0006/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:30,169 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local204610770_0006/job_local204610770_0006.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:30,170 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local204610770_0006/job_local204610770_0006.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:30,171 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:30,171 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:30,171 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local204610770_0006
2014-04-10 11:46:30,171 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:30,171 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2014-04-10 11:46:30,174 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:30,174 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local204610770_0006_m_000000_0
2014-04-10 11:46:30,176 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:30,176 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-brock/8506900574_count_412DBAF5170A4C4789F5E0FD5DC2386A/part-00000
2014-04-10 11:46:30,176 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@63ed4b47
2014-04-10 11:46:30,182 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:30,182 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:30,193 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:30,194 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:30,194 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:30,194 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:30,194 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:30,195 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:30,195 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:30,200 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: TempHfs["SequenceFile[['count', 'count2']]"][8506900574/count/]
2014-04-10 11:46:30,200 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(count)[by:[{1}:'count']]
2014-04-10 11:46:30,201 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:30,201 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:30,201 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:30,202 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 21; bufvoid = 104857600
2014-04-10 11:46:30,202 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214388(104857552); length = 9/6553600
2014-04-10 11:46:30,203 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:30,204 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local204610770_0006_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:30,205 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-brock/8506900574_count_412DBAF5170A4C4789F5E0FD5DC2386A/part-00000:0+112
2014-04-10 11:46:30,205 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local204610770_0006_m_000000_0' done.
2014-04-10 11:46:30,205 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local204610770_0006_m_000000_0
2014-04-10 11:46:30,205 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:30,206 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:30,206 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local204610770_0006_r_000000_0
2014-04-10 11:46:30,207 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:30,207 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@23a86a12
2014-04-10 11:46:30,207 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:30,207 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local204610770_0006_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:30,209 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#5 about to shuffle output of map attempt_local204610770_0006_m_000000_0 decomp: 29 len: 33 to MEMORY
2014-04-10 11:46:30,209 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 29 bytes from map-output for attempt_local204610770_0006_m_000000_0
2014-04-10 11:46:30,209 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 29, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->29
2014-04-10 11:46:30,210 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:30,210 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:30,210 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:30,211 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:30,212 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 24 bytes
2014-04-10 11:46:30,212 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 29 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:30,213 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 33 bytes from disk
2014-04-10 11:46:30,213 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:30,213 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:30,213 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 24 bytes
2014-04-10 11:46:30,213 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:30,214 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:30,214 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:30,220 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(count)[by:[{1}:'count']]
2014-04-10 11:46:30,220 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['count', 'count2']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount"]
2014-04-10 11:46:30,227 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local204610770_0006_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:30,229 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:30,229 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local204610770_0006_r_000000_0 is allowed to commit now
2014-04-10 11:46:30,230 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local204610770_0006_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount/_temporary/0/task_local204610770_0006_r_000000
2014-04-10 11:46:30,231 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:30,231 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local204610770_0006_r_000000_0' done.
2014-04-10 11:46:30,232 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local204610770_0006_r_000000_0
2014-04-10 11:46:30,232 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:30,255 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount/_temporary
2014-04-10 11:46:30,273 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:30,290 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:30,292 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/countcount
2014-04-10 11:46:30,306 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:30,307 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:30,314 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:30,316 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:30,317 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:30,323 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs+count-rhs] starting
2014-04-10 11:46:30,323 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs+count-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt"]
2014-04-10 11:46:30,323 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs+count-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:30,324 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs+count-rhs]  sink: Hfs["TextDelimited[['char', 'count']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount"]
2014-04-10 11:46:30,324 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs+count-rhs]  parallel execution is enabled: false
2014-04-10 11:46:30,324 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs+count-rhs]  starting jobs: 1
2014-04-10 11:46:30,324 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs+count-rhs]  allocating threads: 1
2014-04-10 11:46:30,324 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs+count-rhs] starting step: (1/1) ...elpersplatform/mergecount
2014-04-10 11:46:30,325 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:30,326 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:30,345 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:30,347 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:30,360 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2014-04-10 11:46:30,373 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1984907562_0007
2014-04-10 11:46:30,385 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1984907562/.staging/job_local1984907562_0007/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:30,385 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1984907562/.staging/job_local1984907562_0007/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:30,415 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1984907562_0007/job_local1984907562_0007.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:30,415 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1984907562_0007/job_local1984907562_0007.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:30,415 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:30,416 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs+count-rhs] submitted hadoop job: job_local1984907562_0007
2014-04-10 11:46:30,416 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:30,416 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs+count-rhs] tracking url: http://localhost:8080/
2014-04-10 11:46:30,416 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:30,418 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:30,418 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1984907562_0007_m_000000_0
2014-04-10 11:46:30,421 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:30,422 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:30,423 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1587be37
2014-04-10 11:46:30,424 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:30,425 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:30,437 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:30,437 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:30,437 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:30,437 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:30,437 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:30,438 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:30,438 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:30,444 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:30,444 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2014-04-10 11:46:30,444 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:30,445 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:30,445 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 250, total: 413, max: 672
2014-04-10 11:46:30,445 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 61% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:30,446 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:30,446 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 250, total: 413, max: 672
2014-04-10 11:46:30,446 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 61% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:30,446 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-04-10 11:46:30,446 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 250, total: 413, max: 672
2014-04-10 11:46:30,446 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 61% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:30,447 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-04-10 11:46:30,447 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 250, total: 413, max: 672
2014-04-10 11:46:30,447 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 61% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:30,448 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:30,448 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:30,448 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:30,448 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2014-04-10 11:46:30,448 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2014-04-10 11:46:30,450 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:30,451 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1984907562_0007_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:30,452 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:30,452 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1984907562_0007_m_000000_0' done.
2014-04-10 11:46:30,452 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1984907562_0007_m_000000_0
2014-04-10 11:46:30,452 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1984907562_0007_m_000001_0
2014-04-10 11:46:30,453 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:30,454 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt
2014-04-10 11:46:30,454 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1ba83dc
2014-04-10 11:46:30,454 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:30,455 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:30,466 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:30,466 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:30,466 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:30,466 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:30,466 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:30,467 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:30,467 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:30,473 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt"]
2014-04-10 11:46:30,473 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2014-04-10 11:46:30,473 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:30,475 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:30,475 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 148, total: 413, max: 672
2014-04-10 11:46:30,475 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 61% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:30,476 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:30,476 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 148, total: 413, max: 672
2014-04-10 11:46:30,476 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 61% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:30,476 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-04-10 11:46:30,476 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 148, total: 413, max: 672
2014-04-10 11:46:30,476 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 61% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:30,477 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-04-10 11:46:30,477 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 148, total: 413, max: 672
2014-04-10 11:46:30,477 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 61% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:30,478 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:30,478 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:30,478 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:30,478 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2014-04-10 11:46:30,478 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2014-04-10 11:46:30,480 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:30,482 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1984907562_0007_m_000001_0 is done. And is in the process of committing
2014-04-10 11:46:30,484 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt:0+51
2014-04-10 11:46:30,484 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1984907562_0007_m_000001_0' done.
2014-04-10 11:46:30,484 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1984907562_0007_m_000001_0
2014-04-10 11:46:30,485 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:30,485 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:30,485 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1984907562_0007_r_000000_0
2014-04-10 11:46:30,488 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:30,488 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@44eb0612
2014-04-10 11:46:30,489 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:30,489 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1984907562_0007_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:30,492 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#6 about to shuffle output of map attempt_local1984907562_0007_m_000000_0 decomp: 119 len: 123 to MEMORY
2014-04-10 11:46:30,493 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local1984907562_0007_m_000000_0
2014-04-10 11:46:30,493 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2014-04-10 11:46:30,496 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#6 about to shuffle output of map attempt_local1984907562_0007_m_000001_0 decomp: 119 len: 123 to MEMORY
2014-04-10 11:46:30,497 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local1984907562_0007_m_000001_0
2014-04-10 11:46:30,497 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 2, commitMemory -> 119, usedMemory ->238
2014-04-10 11:46:30,497 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:30,498 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:30,498 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:30,500 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2014-04-10 11:46:30,500 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 220 bytes
2014-04-10 11:46:30,502 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 238 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:30,503 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 240 bytes from disk
2014-04-10 11:46:30,503 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:30,503 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:30,503 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 227 bytes
2014-04-10 11:46:30,504 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:30,504 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:30,504 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:30,513 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2014-04-10 11:46:30,514 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount"]
2014-04-10 11:46:30,520 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1984907562_0007_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:30,521 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:30,521 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1984907562_0007_r_000000_0 is allowed to commit now
2014-04-10 11:46:30,522 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1984907562_0007_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount/_temporary/0/task_local1984907562_0007_r_000000
2014-04-10 11:46:30,523 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:30,523 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1984907562_0007_r_000000_0' done.
2014-04-10 11:46:30,523 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1984907562_0007_r_000000_0
2014-04-10 11:46:30,523 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:30,533 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount/_temporary
2014-04-10 11:46:30,552 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:30,576 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:30,578 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/mergecount
2014-04-10 11:46:30,596 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:30,596 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:30,601 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:30,602 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:30,602 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:30,608 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [max] starting
2014-04-10 11:46:30,609 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [max]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:30,609 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [max]  sink: Hfs["TextDelimited[['num', 'max']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring"]
2014-04-10 11:46:30,609 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [max]  parallel execution is enabled: false
2014-04-10 11:46:30,609 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [max]  starting jobs: 1
2014-04-10 11:46:30,609 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [max]  allocating threads: 1
2014-04-10 11:46:30,609 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...lpersplatform/maxbystring
2014-04-10 11:46:30,610 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:30,610 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:30,619 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:30,624 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:30,633 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1808798164_0008
2014-04-10 11:46:30,644 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1808798164/.staging/job_local1808798164_0008/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:30,644 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1808798164/.staging/job_local1808798164_0008/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:30,670 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1808798164_0008/job_local1808798164_0008.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:30,670 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1808798164_0008/job_local1808798164_0008.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:30,670 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:30,671 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local1808798164_0008
2014-04-10 11:46:30,671 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:30,671 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2014-04-10 11:46:30,671 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:30,674 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:30,674 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1808798164_0008_m_000000_0
2014-04-10 11:46:30,675 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:30,676 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:30,676 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@545b14fd
2014-04-10 11:46:30,677 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:30,677 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:30,691 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:30,691 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:30,691 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:30,691 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:30,691 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:30,692 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:30,692 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:30,697 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:30,697 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(max)[by:[{1}:'num']]
2014-04-10 11:46:30,697 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:30,698 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:30,698 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 155, total: 413, max: 672
2014-04-10 11:46:30,698 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 61% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:30,698 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:30,699 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 155, total: 413, max: 672
2014-04-10 11:46:30,699 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 61% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:30,699 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:30,699 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:30,699 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:30,700 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2014-04-10 11:46:30,700 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2014-04-10 11:46:30,701 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:30,703 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1808798164_0008_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:30,705 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:30,706 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1808798164_0008_m_000000_0' done.
2014-04-10 11:46:30,706 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1808798164_0008_m_000000_0
2014-04-10 11:46:30,706 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:30,707 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:30,707 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1808798164_0008_r_000000_0
2014-04-10 11:46:30,708 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:30,708 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@290d1875
2014-04-10 11:46:30,708 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:30,709 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1808798164_0008_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:30,710 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#7 about to shuffle output of map attempt_local1808798164_0008_m_000000_0 decomp: 87 len: 91 to MEMORY
2014-04-10 11:46:30,711 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local1808798164_0008_m_000000_0
2014-04-10 11:46:30,711 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2014-04-10 11:46:30,712 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:30,713 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:30,713 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:30,714 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:30,714 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2014-04-10 11:46:30,716 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 87 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:30,716 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 91 bytes from disk
2014-04-10 11:46:30,716 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:30,716 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:30,717 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2014-04-10 11:46:30,717 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:30,718 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:30,718 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:30,727 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(max)[by:[{1}:'num']]
2014-04-10 11:46:30,727 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['num', 'max']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring"]
2014-04-10 11:46:30,733 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1808798164_0008_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:30,733 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:30,734 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1808798164_0008_r_000000_0 is allowed to commit now
2014-04-10 11:46:30,734 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1808798164_0008_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring/_temporary/0/task_local1808798164_0008_r_000000
2014-04-10 11:46:30,735 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:30,735 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1808798164_0008_r_000000_0' done.
2014-04-10 11:46:30,735 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1808798164_0008_r_000000_0
2014-04-10 11:46:30,735 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:30,746 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring/_temporary
2014-04-10 11:46:30,759 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:30,774 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:30,775 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/maxbystring
2014-04-10 11:46:30,787 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:30,788 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:30,792 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:30,793 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:30,793 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:30,802 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [first] starting
2014-04-10 11:46:30,802 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [first]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:30,802 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [first]  sink: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstn"]
2014-04-10 11:46:30,802 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [first]  parallel execution is enabled: false
2014-04-10 11:46:30,802 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [first]  starting jobs: 1
2014-04-10 11:46:30,802 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [first]  allocating threads: 1
2014-04-10 11:46:30,802 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] starting step: (1/1) ...blyhelpersplatform/firstn
2014-04-10 11:46:30,803 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:30,803 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:30,812 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:30,818 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:30,824 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1430060057_0009
2014-04-10 11:46:30,833 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1430060057/.staging/job_local1430060057_0009/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:30,833 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1430060057/.staging/job_local1430060057_0009/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:30,856 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1430060057_0009/job_local1430060057_0009.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:30,857 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1430060057_0009/job_local1430060057_0009.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:30,857 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:30,857 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:30,857 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] submitted hadoop job: job_local1430060057_0009
2014-04-10 11:46:30,858 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:30,858 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] tracking url: http://localhost:8080/
2014-04-10 11:46:30,859 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:30,859 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1430060057_0009_m_000000_0
2014-04-10 11:46:30,860 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:30,861 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:30,861 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@75885dd3
2014-04-10 11:46:30,861 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:30,862 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:30,872 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:30,872 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:30,873 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:30,873 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:30,873 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:30,874 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:30,874 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:30,878 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:30,878 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(first)[by:[{1}:'num']]
2014-04-10 11:46:30,879 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:30,880 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:30,880 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 198, total: 455, max: 672
2014-04-10 11:46:30,880 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 67% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:30,880 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:30,880 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 198, total: 455, max: 672
2014-04-10 11:46:30,880 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 67% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:30,881 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:30,881 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:30,881 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:30,881 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 110; bufvoid = 104857600
2014-04-10 11:46:30,881 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2014-04-10 11:46:30,883 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:30,884 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1430060057_0009_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:30,887 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:30,887 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1430060057_0009_m_000000_0' done.
2014-04-10 11:46:30,887 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1430060057_0009_m_000000_0
2014-04-10 11:46:30,887 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:30,888 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:30,888 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1430060057_0009_r_000000_0
2014-04-10 11:46:30,889 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:30,889 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@42c08a7e
2014-04-10 11:46:30,889 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:30,890 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1430060057_0009_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:30,890 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#8 about to shuffle output of map attempt_local1430060057_0009_m_000000_0 decomp: 122 len: 126 to MEMORY
2014-04-10 11:46:30,891 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 122 bytes from map-output for attempt_local1430060057_0009_m_000000_0
2014-04-10 11:46:30,891 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 122, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->122
2014-04-10 11:46:30,891 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:30,891 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:30,892 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:30,893 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:30,893 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 106 bytes
2014-04-10 11:46:30,893 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 122 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:30,893 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 126 bytes from disk
2014-04-10 11:46:30,894 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:30,894 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:30,894 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 106 bytes
2014-04-10 11:46:30,894 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:30,894 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:30,894 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:30,898 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(first)[by:[{1}:'num']]
2014-04-10 11:46:30,899 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstn"]
2014-04-10 11:46:30,903 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1430060057_0009_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:30,904 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:30,904 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1430060057_0009_r_000000_0 is allowed to commit now
2014-04-10 11:46:30,904 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1430060057_0009_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstn/_temporary/0/task_local1430060057_0009_r_000000
2014-04-10 11:46:30,905 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:30,905 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1430060057_0009_r_000000_0' done.
2014-04-10 11:46:30,905 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1430060057_0009_r_000000_0
2014-04-10 11:46:30,905 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:30,916 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstn/_temporary
2014-04-10 11:46:30,931 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:30,932 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/firstn
2014-04-10 11:46:30,942 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:30,942 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:30,946 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:30,947 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:30,947 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:30,953 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce] starting
2014-04-10 11:46:30,953 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt"]
2014-04-10 11:46:30,954 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce]  sink: Hfs["TextLine[['line']->['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields"]
2014-04-10 11:46:30,954 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce]  parallel execution is enabled: false
2014-04-10 11:46:30,954 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce]  starting jobs: 1
2014-04-10 11:46:30,954 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce]  allocating threads: 1
2014-04-10 11:46:30,955 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] starting step: (1/1) ...persplatform/coercefields
2014-04-10 11:46:30,955 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:30,956 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:30,972 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:30,982 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:30,994 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local2018851838_0010
2014-04-10 11:46:31,005 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock2018851838/.staging/job_local2018851838_0010/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:31,005 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock2018851838/.staging/job_local2018851838_0010/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:31,027 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local2018851838_0010/job_local2018851838_0010.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:31,027 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local2018851838_0010/job_local2018851838_0010.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:31,028 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:31,028 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:31,028 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] submitted hadoop job: job_local2018851838_0010
2014-04-10 11:46:31,028 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:31,028 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] tracking url: http://localhost:8080/
2014-04-10 11:46:31,030 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:31,031 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local2018851838_0010_m_000000_0
2014-04-10 11:46:31,032 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:31,033 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt
2014-04-10 11:46:31,034 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5a78cf3a
2014-04-10 11:46:31,035 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2014-04-10 11:46:31,040 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:31,040 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:31,049 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt"]
2014-04-10 11:46:31,049 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: Hfs["TextLine[['line']->['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields"]
2014-04-10 11:46:31,053 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:31,054 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local2018851838_0010_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:31,054 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:31,054 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local2018851838_0010_m_000000_0 is allowed to commit now
2014-04-10 11:46:31,055 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local2018851838_0010_m_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields/_temporary/0/task_local2018851838_0010_m_000000
2014-04-10 11:46:31,056 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt:0+19
2014-04-10 11:46:31,056 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2018851838_0010_m_000000_0' done.
2014-04-10 11:46:31,056 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local2018851838_0010_m_000000_0
2014-04-10 11:46:31,056 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:31,080 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields/_temporary
2014-04-10 11:46:31,105 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:31,106 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/coercefields
2014-04-10 11:46:31,119 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:31,119 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:31,124 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:31,125 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:31,125 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:31,131 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count] starting
2014-04-10 11:46:31,131 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:31,131 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count]  sink: Hfs["TextDelimited[['char', 'count']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count"]
2014-04-10 11:46:31,131 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count]  parallel execution is enabled: false
2014-04-10 11:46:31,132 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count]  starting jobs: 1
2014-04-10 11:46:31,132 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count]  allocating threads: 1
2014-04-10 11:46:31,132 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...mblyhelpersplatform/count
2014-04-10 11:46:31,132 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:31,133 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:31,147 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:31,159 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:31,170 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1204492698_0011
2014-04-10 11:46:31,183 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1204492698/.staging/job_local1204492698_0011/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:31,183 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1204492698/.staging/job_local1204492698_0011/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:31,207 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1204492698_0011/job_local1204492698_0011.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:31,207 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1204492698_0011/job_local1204492698_0011.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:31,207 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:31,208 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local1204492698_0011
2014-04-10 11:46:31,208 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:31,208 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2014-04-10 11:46:31,208 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:31,211 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:31,211 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1204492698_0011_m_000000_0
2014-04-10 11:46:31,212 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:31,214 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:31,214 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@10f70ce1
2014-04-10 11:46:31,215 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:31,215 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:31,230 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:31,230 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:31,230 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:31,230 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:31,230 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:31,231 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:31,231 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:31,235 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:31,235 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(count)[by:[{1}:'char']]
2014-04-10 11:46:31,235 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:31,236 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:31,236 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 195, total: 454, max: 672
2014-04-10 11:46:31,236 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 67% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:31,237 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:31,237 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 195, total: 454, max: 672
2014-04-10 11:46:31,237 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 67% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:31,237 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-04-10 11:46:31,237 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 195, total: 454, max: 672
2014-04-10 11:46:31,237 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 67% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:31,238 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-04-10 11:46:31,238 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 195, total: 454, max: 672
2014-04-10 11:46:31,238 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 67% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:31,238 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:31,238 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:31,238 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:31,239 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2014-04-10 11:46:31,239 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2014-04-10 11:46:31,240 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:31,241 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1204492698_0011_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:31,242 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:31,242 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1204492698_0011_m_000000_0' done.
2014-04-10 11:46:31,242 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1204492698_0011_m_000000_0
2014-04-10 11:46:31,243 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:31,243 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:31,243 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1204492698_0011_r_000000_0
2014-04-10 11:46:31,245 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:31,245 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@4e20562e
2014-04-10 11:46:31,246 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:31,246 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1204492698_0011_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:31,248 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#9 about to shuffle output of map attempt_local1204492698_0011_m_000000_0 decomp: 119 len: 123 to MEMORY
2014-04-10 11:46:31,249 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local1204492698_0011_m_000000_0
2014-04-10 11:46:31,249 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2014-04-10 11:46:31,250 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:31,251 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:31,251 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:31,253 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:31,253 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2014-04-10 11:46:31,254 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:31,254 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2014-04-10 11:46:31,254 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:31,255 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:31,255 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2014-04-10 11:46:31,256 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:31,256 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:31,256 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:31,265 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(count)[by:[{1}:'char']]
2014-04-10 11:46:31,266 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count"]
2014-04-10 11:46:31,273 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1204492698_0011_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:31,276 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:31,276 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1204492698_0011_r_000000_0 is allowed to commit now
2014-04-10 11:46:31,277 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1204492698_0011_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count/_temporary/0/task_local1204492698_0011_r_000000
2014-04-10 11:46:31,278 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:31,278 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1204492698_0011_r_000000_0' done.
2014-04-10 11:46:31,279 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1204492698_0011_r_000000_0
2014-04-10 11:46:31,279 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:31,299 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count/_temporary
2014-04-10 11:46:31,309 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:31,323 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:31,324 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/count
2014-04-10 11:46:31,341 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:31,342 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:31,345 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:31,346 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:31,347 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:31,351 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [max] starting
2014-04-10 11:46:31,351 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [max]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:31,351 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [max]  sink: Hfs["TextDelimited[['char', 'max']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby"]
2014-04-10 11:46:31,352 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [max]  parallel execution is enabled: false
2014-04-10 11:46:31,352 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [max]  starting jobs: 1
2014-04-10 11:46:31,352 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [max]  allocating threads: 1
2014-04-10 11:46:31,352 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...mblyhelpersplatform/maxby
2014-04-10 11:46:31,353 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:31,354 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:31,369 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:31,379 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:31,390 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local62977773_0012
2014-04-10 11:46:31,402 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock62977773/.staging/job_local62977773_0012/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:31,402 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock62977773/.staging/job_local62977773_0012/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:31,432 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local62977773_0012/job_local62977773_0012.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:31,432 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local62977773_0012/job_local62977773_0012.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:31,433 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:31,433 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local62977773_0012
2014-04-10 11:46:31,433 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2014-04-10 11:46:31,433 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:31,434 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:31,436 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:31,436 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local62977773_0012_m_000000_0
2014-04-10 11:46:31,438 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:31,439 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:31,439 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@296ba81d
2014-04-10 11:46:31,441 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:31,441 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:31,459 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:31,460 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:31,460 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:31,460 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:31,460 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:31,463 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:31,463 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:31,472 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:31,473 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(max)[by:[{1}:'char']]
2014-04-10 11:46:31,473 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:31,475 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:31,475 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 245, total: 459, max: 672
2014-04-10 11:46:31,475 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:31,476 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:31,476 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 245, total: 459, max: 672
2014-04-10 11:46:31,476 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:31,476 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-04-10 11:46:31,477 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 245, total: 459, max: 672
2014-04-10 11:46:31,477 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:31,477 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-04-10 11:46:31,477 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 245, total: 459, max: 672
2014-04-10 11:46:31,478 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:31,478 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:31,479 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:31,479 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:31,479 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 135; bufvoid = 104857600
2014-04-10 11:46:31,479 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2014-04-10 11:46:31,480 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:31,481 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local62977773_0012_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:31,482 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:31,482 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local62977773_0012_m_000000_0' done.
2014-04-10 11:46:31,482 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local62977773_0012_m_000000_0
2014-04-10 11:46:31,482 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:31,483 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:31,483 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local62977773_0012_r_000000_0
2014-04-10 11:46:31,484 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:31,485 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@906d2b4
2014-04-10 11:46:31,485 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:31,486 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local62977773_0012_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:31,488 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#10 about to shuffle output of map attempt_local62977773_0012_m_000000_0 decomp: 155 len: 159 to MEMORY
2014-04-10 11:46:31,488 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 155 bytes from map-output for attempt_local62977773_0012_m_000000_0
2014-04-10 11:46:31,488 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 155, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->155
2014-04-10 11:46:31,489 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:31,490 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:31,490 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:31,492 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:31,492 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2014-04-10 11:46:31,493 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 155 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:31,494 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 159 bytes from disk
2014-04-10 11:46:31,494 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:31,494 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:31,494 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2014-04-10 11:46:31,495 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:31,496 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:31,496 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:31,501 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(max)[by:[{1}:'char']]
2014-04-10 11:46:31,501 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'max']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby"]
2014-04-10 11:46:31,508 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local62977773_0012_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:31,509 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:31,510 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local62977773_0012_r_000000_0 is allowed to commit now
2014-04-10 11:46:31,510 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local62977773_0012_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby/_temporary/0/task_local62977773_0012_r_000000
2014-04-10 11:46:31,512 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:31,512 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local62977773_0012_r_000000_0' done.
2014-04-10 11:46:31,512 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local62977773_0012_r_000000_0
2014-04-10 11:46:31,512 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:31,532 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby/_temporary
2014-04-10 11:46:31,555 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:31,576 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:31,577 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/maxby
2014-04-10 11:46:31,592 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:31,593 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:31,596 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:31,597 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:31,598 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:31,602 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [min] starting
2014-04-10 11:46:31,602 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [min]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:31,603 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [min]  sink: Hfs["TextDelimited[['char', 'min']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby"]
2014-04-10 11:46:31,603 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [min]  parallel execution is enabled: false
2014-04-10 11:46:31,603 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [min]  starting jobs: 1
2014-04-10 11:46:31,603 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [min]  allocating threads: 1
2014-04-10 11:46:31,604 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] starting step: (1/1) ...mblyhelpersplatform/minby
2014-04-10 11:46:31,605 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:31,606 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:31,620 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:31,630 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:31,641 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1516955697_0013
2014-04-10 11:46:31,652 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1516955697/.staging/job_local1516955697_0013/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:31,653 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1516955697/.staging/job_local1516955697_0013/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:31,672 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1516955697_0013/job_local1516955697_0013.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:31,672 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1516955697_0013/job_local1516955697_0013.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:31,673 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:31,673 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] submitted hadoop job: job_local1516955697_0013
2014-04-10 11:46:31,673 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] tracking url: http://localhost:8080/
2014-04-10 11:46:31,673 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:31,674 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:31,676 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:31,676 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1516955697_0013_m_000000_0
2014-04-10 11:46:31,677 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:31,678 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:31,678 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@33932b25
2014-04-10 11:46:31,679 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:31,679 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:31,692 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:31,694 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:31,694 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:31,694 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:31,695 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:31,695 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:31,695 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:31,699 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:31,700 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(min)[by:[{1}:'char']]
2014-04-10 11:46:31,700 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:31,700 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:31,701 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 245, total: 458, max: 672
2014-04-10 11:46:31,701 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:31,701 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:31,701 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 245, total: 458, max: 672
2014-04-10 11:46:31,701 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:31,701 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-04-10 11:46:31,701 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 245, total: 458, max: 672
2014-04-10 11:46:31,702 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:31,702 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-04-10 11:46:31,702 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 245, total: 458, max: 672
2014-04-10 11:46:31,702 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:31,703 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:31,703 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:31,703 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:31,703 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 135; bufvoid = 104857600
2014-04-10 11:46:31,703 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2014-04-10 11:46:31,704 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:31,705 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1516955697_0013_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:31,706 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:31,706 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1516955697_0013_m_000000_0' done.
2014-04-10 11:46:31,706 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1516955697_0013_m_000000_0
2014-04-10 11:46:31,706 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:31,707 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:31,707 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1516955697_0013_r_000000_0
2014-04-10 11:46:31,708 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:31,708 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@23271ac4
2014-04-10 11:46:31,709 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:31,709 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1516955697_0013_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:31,710 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#11 about to shuffle output of map attempt_local1516955697_0013_m_000000_0 decomp: 155 len: 159 to MEMORY
2014-04-10 11:46:31,711 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 155 bytes from map-output for attempt_local1516955697_0013_m_000000_0
2014-04-10 11:46:31,711 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 155, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->155
2014-04-10 11:46:31,711 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:31,712 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:31,712 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:31,713 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:31,713 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2014-04-10 11:46:31,714 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 155 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:31,714 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 159 bytes from disk
2014-04-10 11:46:31,714 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:31,714 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:31,714 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2014-04-10 11:46:31,714 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:31,715 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:31,715 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:31,718 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(min)[by:[{1}:'char']]
2014-04-10 11:46:31,718 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'min']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby"]
2014-04-10 11:46:31,724 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1516955697_0013_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:31,726 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:31,726 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1516955697_0013_r_000000_0 is allowed to commit now
2014-04-10 11:46:31,727 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1516955697_0013_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby/_temporary/0/task_local1516955697_0013_r_000000
2014-04-10 11:46:31,729 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:31,729 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1516955697_0013_r_000000_0' done.
2014-04-10 11:46:31,729 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1516955697_0013_r_000000_0
2014-04-10 11:46:31,729 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:31,750 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby/_temporary
2014-04-10 11:46:31,773 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:31,795 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:31,795 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/minby
2014-04-10 11:46:31,808 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:31,808 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:31,812 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:31,813 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:31,813 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:31,818 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [sum] starting
2014-04-10 11:46:31,818 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [sum]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:31,818 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [sum]  sink: Hfs["TextDelimited[['char', 'sum']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum"]
2014-04-10 11:46:31,819 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [sum]  parallel execution is enabled: false
2014-04-10 11:46:31,819 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [sum]  starting jobs: 1
2014-04-10 11:46:31,819 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [sum]  allocating threads: 1
2014-04-10 11:46:31,820 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] starting step: (1/1) ...semblyhelpersplatform/sum
2014-04-10 11:46:31,820 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:31,821 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:31,835 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:31,845 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:31,855 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local346052531_0014
2014-04-10 11:46:31,871 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock346052531/.staging/job_local346052531_0014/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:31,871 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock346052531/.staging/job_local346052531_0014/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:31,890 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local346052531_0014/job_local346052531_0014.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:31,890 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local346052531_0014/job_local346052531_0014.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:31,891 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:31,891 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] submitted hadoop job: job_local346052531_0014
2014-04-10 11:46:31,891 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] tracking url: http://localhost:8080/
2014-04-10 11:46:31,891 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:31,892 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:31,894 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:31,894 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local346052531_0014_m_000000_0
2014-04-10 11:46:31,896 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:31,897 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:31,897 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@442d2ee0
2014-04-10 11:46:31,899 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:31,899 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:31,917 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:31,918 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:31,919 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:31,919 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:31,919 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:31,920 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:31,920 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:31,927 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:31,927 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(sum)[by:[{1}:'char']]
2014-04-10 11:46:31,928 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:31,928 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:31,929 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 248, total: 461, max: 672
2014-04-10 11:46:31,929 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:31,929 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:31,929 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 248, total: 461, max: 672
2014-04-10 11:46:31,929 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:31,930 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-04-10 11:46:31,930 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 248, total: 461, max: 672
2014-04-10 11:46:31,930 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:31,930 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-04-10 11:46:31,930 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 248, total: 461, max: 672
2014-04-10 11:46:31,930 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:31,931 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:31,931 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:31,931 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:31,932 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2014-04-10 11:46:31,932 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2014-04-10 11:46:31,934 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:31,935 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local346052531_0014_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:31,936 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:31,936 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local346052531_0014_m_000000_0' done.
2014-04-10 11:46:31,936 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local346052531_0014_m_000000_0
2014-04-10 11:46:31,937 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:31,937 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:31,937 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local346052531_0014_r_000000_0
2014-04-10 11:46:31,938 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:31,938 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5f242a67
2014-04-10 11:46:31,939 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:31,939 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local346052531_0014_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:31,940 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#12 about to shuffle output of map attempt_local346052531_0014_m_000000_0 decomp: 119 len: 123 to MEMORY
2014-04-10 11:46:31,940 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local346052531_0014_m_000000_0
2014-04-10 11:46:31,940 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2014-04-10 11:46:31,940 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:31,941 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:31,941 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:31,942 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:31,942 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2014-04-10 11:46:31,942 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:31,942 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2014-04-10 11:46:31,942 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:31,943 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:31,943 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2014-04-10 11:46:31,943 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:31,943 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:31,943 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:31,948 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(sum)[by:[{1}:'char']]
2014-04-10 11:46:31,948 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum"]
2014-04-10 11:46:31,952 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local346052531_0014_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:31,954 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:31,954 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local346052531_0014_r_000000_0 is allowed to commit now
2014-04-10 11:46:31,955 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local346052531_0014_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum/_temporary/0/task_local346052531_0014_r_000000
2014-04-10 11:46:31,956 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:31,956 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local346052531_0014_r_000000_0' done.
2014-04-10 11:46:31,957 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local346052531_0014_r_000000_0
2014-04-10 11:46:31,957 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:31,977 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum/_temporary
2014-04-10 11:46:31,998 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:32,017 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:32,018 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/sum
2014-04-10 11:46:32,030 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:32,031 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:32,036 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:32,037 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:32,038 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:32,043 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [sum-lhs+sum-rhs] starting
2014-04-10 11:46:32,043 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [sum-lhs+sum-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt"]
2014-04-10 11:46:32,043 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [sum-lhs+sum-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:32,043 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [sum-lhs+sum-rhs]  sink: Hfs["TextDelimited[['char', 'sum']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum"]
2014-04-10 11:46:32,043 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [sum-lhs+sum-rhs]  parallel execution is enabled: false
2014-04-10 11:46:32,044 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [sum-lhs+sum-rhs]  starting jobs: 1
2014-04-10 11:46:32,044 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [sum-lhs+sum-rhs]  allocating threads: 1
2014-04-10 11:46:32,044 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum-lhs+sum-rhs] starting step: (1/1) ...yhelpersplatform/mergesum
2014-04-10 11:46:32,045 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:32,045 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:32,057 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:32,058 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:32,068 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2014-04-10 11:46:32,078 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local103097923_0015
2014-04-10 11:46:32,091 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock103097923/.staging/job_local103097923_0015/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:32,091 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock103097923/.staging/job_local103097923_0015/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:32,111 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local103097923_0015/job_local103097923_0015.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:32,111 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local103097923_0015/job_local103097923_0015.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:32,112 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:32,112 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum-lhs+sum-rhs] submitted hadoop job: job_local103097923_0015
2014-04-10 11:46:32,112 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:32,112 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum-lhs+sum-rhs] tracking url: http://localhost:8080/
2014-04-10 11:46:32,113 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:32,115 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:32,115 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local103097923_0015_m_000000_0
2014-04-10 11:46:32,116 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:32,117 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:32,118 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4574cdb2
2014-04-10 11:46:32,119 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:32,119 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:32,137 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:32,138 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:32,138 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:32,138 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:32,138 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:32,139 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:32,139 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:32,144 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:32,144 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2014-04-10 11:46:32,144 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:32,144 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:32,144 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 247, total: 460, max: 672
2014-04-10 11:46:32,145 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:32,145 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:32,145 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 247, total: 460, max: 672
2014-04-10 11:46:32,145 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:32,145 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-04-10 11:46:32,145 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 247, total: 460, max: 672
2014-04-10 11:46:32,145 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:32,146 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-04-10 11:46:32,146 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 247, total: 460, max: 672
2014-04-10 11:46:32,146 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:32,146 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:32,146 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:32,147 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:32,147 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2014-04-10 11:46:32,147 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2014-04-10 11:46:32,148 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:32,149 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local103097923_0015_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:32,150 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:32,150 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local103097923_0015_m_000000_0' done.
2014-04-10 11:46:32,150 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local103097923_0015_m_000000_0
2014-04-10 11:46:32,150 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local103097923_0015_m_000001_0
2014-04-10 11:46:32,151 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:32,151 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt
2014-04-10 11:46:32,151 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@475dd215
2014-04-10 11:46:32,152 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:32,152 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:32,162 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:32,162 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:32,162 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:32,162 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:32,162 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:32,163 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:32,163 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:32,168 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt"]
2014-04-10 11:46:32,168 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2014-04-10 11:46:32,168 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:32,170 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:32,170 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 146, total: 460, max: 672
2014-04-10 11:46:32,170 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:32,171 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:32,171 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 146, total: 460, max: 672
2014-04-10 11:46:32,171 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:32,171 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-04-10 11:46:32,171 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 146, total: 460, max: 672
2014-04-10 11:46:32,171 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:32,171 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-04-10 11:46:32,171 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 146, total: 460, max: 672
2014-04-10 11:46:32,171 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:32,172 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:32,172 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:32,172 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:32,172 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2014-04-10 11:46:32,172 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2014-04-10 11:46:32,174 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:32,174 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local103097923_0015_m_000001_0 is done. And is in the process of committing
2014-04-10 11:46:32,175 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt:0+51
2014-04-10 11:46:32,175 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local103097923_0015_m_000001_0' done.
2014-04-10 11:46:32,176 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local103097923_0015_m_000001_0
2014-04-10 11:46:32,176 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:32,176 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:32,177 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local103097923_0015_r_000000_0
2014-04-10 11:46:32,179 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:32,179 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@32cc9262
2014-04-10 11:46:32,180 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:32,180 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local103097923_0015_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:32,182 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#13 about to shuffle output of map attempt_local103097923_0015_m_000000_0 decomp: 119 len: 123 to MEMORY
2014-04-10 11:46:32,182 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local103097923_0015_m_000000_0
2014-04-10 11:46:32,182 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2014-04-10 11:46:32,183 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#13 about to shuffle output of map attempt_local103097923_0015_m_000001_0 decomp: 119 len: 123 to MEMORY
2014-04-10 11:46:32,183 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local103097923_0015_m_000001_0
2014-04-10 11:46:32,184 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 2, commitMemory -> 119, usedMemory ->238
2014-04-10 11:46:32,184 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:32,185 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:32,185 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:32,187 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2014-04-10 11:46:32,187 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 220 bytes
2014-04-10 11:46:32,189 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 238 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:32,189 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 240 bytes from disk
2014-04-10 11:46:32,189 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:32,190 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:32,190 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 227 bytes
2014-04-10 11:46:32,191 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:32,191 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:32,192 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:32,199 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2014-04-10 11:46:32,200 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum"]
2014-04-10 11:46:32,204 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local103097923_0015_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:32,205 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:32,205 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local103097923_0015_r_000000_0 is allowed to commit now
2014-04-10 11:46:32,205 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local103097923_0015_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum/_temporary/0/task_local103097923_0015_r_000000
2014-04-10 11:46:32,206 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:32,206 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local103097923_0015_r_000000_0' done.
2014-04-10 11:46:32,206 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local103097923_0015_r_000000_0
2014-04-10 11:46:32,206 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:32,228 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum/_temporary
2014-04-10 11:46:32,248 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:32,270 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:32,271 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/mergesum
2014-04-10 11:46:32,286 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:32,287 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:32,290 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:32,291 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:32,291 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:32,295 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] starting
2014-04-10 11:46:32,295 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt"]
2014-04-10 11:46:32,295 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  sink: Hfs["TextLine[['num']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow"]
2014-04-10 11:46:32,296 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  parallel execution is enabled: false
2014-04-10 11:46:32,296 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  starting jobs: 1
2014-04-10 11:46:32,296 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  allocating threads: 1
2014-04-10 11:46:32,296 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...ersplatform/discardnarrow
2014-04-10 11:46:32,297 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:32,298 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:32,311 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:32,320 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:32,331 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1372710685_0016
2014-04-10 11:46:32,343 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1372710685/.staging/job_local1372710685_0016/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:32,343 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1372710685/.staging/job_local1372710685_0016/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:32,366 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1372710685_0016/job_local1372710685_0016.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:32,366 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1372710685_0016/job_local1372710685_0016.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:32,367 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:32,367 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1372710685_0016
2014-04-10 11:46:32,367 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:32,367 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2014-04-10 11:46:32,367 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:32,369 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:32,369 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1372710685_0016_m_000000_0
2014-04-10 11:46:32,371 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:32,372 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt
2014-04-10 11:46:32,372 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2e123812
2014-04-10 11:46:32,373 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2014-04-10 11:46:32,377 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:32,378 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:32,383 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt"]
2014-04-10 11:46:32,384 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: Hfs["TextLine[['num']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow"]
2014-04-10 11:46:32,387 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:32,387 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1372710685_0016_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:32,389 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:32,389 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1372710685_0016_m_000000_0 is allowed to commit now
2014-04-10 11:46:32,389 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1372710685_0016_m_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow/_temporary/0/task_local1372710685_0016_m_000000
2014-04-10 11:46:32,390 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt:0+19
2014-04-10 11:46:32,390 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1372710685_0016_m_000000_0' done.
2014-04-10 11:46:32,390 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1372710685_0016_m_000000_0
2014-04-10 11:46:32,390 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:32,401 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow/_temporary
2014-04-10 11:46:32,422 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:32,424 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/discardnarrow
2014-04-10 11:46:32,440 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:32,441 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:32,443 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:32,444 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:32,444 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:32,447 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] starting
2014-04-10 11:46:32,448 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt"]
2014-04-10 11:46:32,448 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  sink: Hfs["TextLine[['line']->['item', 'element']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename"]
2014-04-10 11:46:32,448 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  parallel execution is enabled: false
2014-04-10 11:46:32,448 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  starting jobs: 1
2014-04-10 11:46:32,448 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  allocating threads: 1
2014-04-10 11:46:32,449 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...blyhelpersplatform/rename
2014-04-10 11:46:32,450 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:32,450 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:32,462 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:32,472 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:32,483 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1680003163_0017
2014-04-10 11:46:32,493 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1680003163/.staging/job_local1680003163_0017/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:32,493 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1680003163/.staging/job_local1680003163_0017/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:32,512 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1680003163_0017/job_local1680003163_0017.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:32,512 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1680003163_0017/job_local1680003163_0017.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:32,513 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:32,513 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1680003163_0017
2014-04-10 11:46:32,513 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:32,513 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2014-04-10 11:46:32,513 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:32,516 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:32,516 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1680003163_0017_m_000000_0
2014-04-10 11:46:32,517 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:32,518 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt
2014-04-10 11:46:32,518 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2212c414
2014-04-10 11:46:32,519 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2014-04-10 11:46:32,522 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:32,522 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:32,526 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt"]
2014-04-10 11:46:32,526 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: Hfs["TextLine[['line']->['item', 'element']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename"]
2014-04-10 11:46:32,528 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:32,528 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1680003163_0017_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:32,530 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:32,530 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1680003163_0017_m_000000_0 is allowed to commit now
2014-04-10 11:46:32,531 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1680003163_0017_m_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename/_temporary/0/task_local1680003163_0017_m_000000
2014-04-10 11:46:32,532 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt:0+19
2014-04-10 11:46:32,532 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1680003163_0017_m_000000_0' done.
2014-04-10 11:46:32,532 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1680003163_0017_m_000000_0
2014-04-10 11:46:32,532 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:32,548 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename/_temporary
2014-04-10 11:46:32,560 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:32,562 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/rename
2014-04-10 11:46:32,575 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:32,576 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:32,584 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:32,586 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:32,586 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:32,592 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [name] starting
2014-04-10 11:46:32,592 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [name]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt"]
2014-04-10 11:46:32,592 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [name]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:32,592 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [name]  sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true"]
2014-04-10 11:46:32,592 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [name]  parallel execution is enabled: false
2014-04-10 11:46:32,592 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [name]  starting jobs: 1
2014-04-10 11:46:32,592 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [name]  allocating threads: 1
2014-04-10 11:46:32,593 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] starting step: (1/1) ...splatform/multimerge+true
2014-04-10 11:46:32,593 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:32,594 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:32,608 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:32,609 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:32,619 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2014-04-10 11:46:32,630 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1962218496_0018
2014-04-10 11:46:32,642 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1962218496/.staging/job_local1962218496_0018/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:32,643 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1962218496/.staging/job_local1962218496_0018/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:32,661 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1962218496_0018/job_local1962218496_0018.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:32,662 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1962218496_0018/job_local1962218496_0018.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:32,662 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:32,663 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] submitted hadoop job: job_local1962218496_0018
2014-04-10 11:46:32,663 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:32,663 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:32,663 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] tracking url: http://localhost:8080/
2014-04-10 11:46:32,665 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:32,665 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1962218496_0018_m_000000_0
2014-04-10 11:46:32,667 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:32,668 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:32,668 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@32e8310d
2014-04-10 11:46:32,669 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:32,669 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:32,687 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:32,688 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:32,688 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:32,689 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:32,689 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:32,691 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:32,691 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:32,703 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:32,703 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2014-04-10 11:46:32,704 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:32,705 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:32,705 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 247, total: 462, max: 672
2014-04-10 11:46:32,705 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:32,706 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:32,706 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 247, total: 462, max: 672
2014-04-10 11:46:32,707 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:32,707 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-04-10 11:46:32,707 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 247, total: 462, max: 672
2014-04-10 11:46:32,708 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:32,708 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-04-10 11:46:32,708 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 247, total: 462, max: 672
2014-04-10 11:46:32,708 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:32,709 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:32,709 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:32,709 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:32,709 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2014-04-10 11:46:32,709 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2014-04-10 11:46:32,711 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:32,711 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1962218496_0018_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:32,712 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:32,712 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1962218496_0018_m_000000_0' done.
2014-04-10 11:46:32,712 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1962218496_0018_m_000000_0
2014-04-10 11:46:32,712 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1962218496_0018_m_000001_0
2014-04-10 11:46:32,714 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:32,715 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt
2014-04-10 11:46:32,715 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2f94abe0
2014-04-10 11:46:32,715 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:32,715 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:32,726 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:32,727 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:32,727 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:32,727 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:32,727 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:32,727 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:32,728 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:32,733 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt"]
2014-04-10 11:46:32,733 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2014-04-10 11:46:32,734 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:32,735 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:32,736 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 145, total: 462, max: 672
2014-04-10 11:46:32,736 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:32,736 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:32,736 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 145, total: 462, max: 672
2014-04-10 11:46:32,736 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:32,737 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-04-10 11:46:32,737 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 145, total: 462, max: 672
2014-04-10 11:46:32,737 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:32,737 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-04-10 11:46:32,737 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 145, total: 462, max: 672
2014-04-10 11:46:32,737 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:32,738 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:32,738 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:32,738 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:32,738 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2014-04-10 11:46:32,738 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2014-04-10 11:46:32,739 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:32,740 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1962218496_0018_m_000001_0 is done. And is in the process of committing
2014-04-10 11:46:32,741 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt:0+51
2014-04-10 11:46:32,741 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1962218496_0018_m_000001_0' done.
2014-04-10 11:46:32,741 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1962218496_0018_m_000001_0
2014-04-10 11:46:32,741 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:32,741 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:32,741 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1962218496_0018_r_000000_0
2014-04-10 11:46:32,743 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:32,743 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@606d3947
2014-04-10 11:46:32,744 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:32,744 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1962218496_0018_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:32,747 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#14 about to shuffle output of map attempt_local1962218496_0018_m_000000_0 decomp: 236 len: 240 to MEMORY
2014-04-10 11:46:32,747 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local1962218496_0018_m_000000_0
2014-04-10 11:46:32,747 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->236
2014-04-10 11:46:32,749 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#14 about to shuffle output of map attempt_local1962218496_0018_m_000001_0 decomp: 236 len: 240 to MEMORY
2014-04-10 11:46:32,749 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local1962218496_0018_m_000001_0
2014-04-10 11:46:32,749 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 2, commitMemory -> 236, usedMemory ->472
2014-04-10 11:46:32,749 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:32,750 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:32,750 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:32,752 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2014-04-10 11:46:32,752 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 454 bytes
2014-04-10 11:46:32,754 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 472 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:32,755 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 474 bytes from disk
2014-04-10 11:46:32,755 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:32,755 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:32,755 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 461 bytes
2014-04-10 11:46:32,756 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:32,757 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:32,757 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:32,768 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(name)[by:[{1}:'char']]
2014-04-10 11:46:32,768 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true"]
2014-04-10 11:46:32,776 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1962218496_0018_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:32,778 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:32,778 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1962218496_0018_r_000000_0 is allowed to commit now
2014-04-10 11:46:32,779 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1962218496_0018_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true/_temporary/0/task_local1962218496_0018_r_000000
2014-04-10 11:46:32,780 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:32,780 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1962218496_0018_r_000000_0' done.
2014-04-10 11:46:32,780 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1962218496_0018_r_000000_0
2014-04-10 11:46:32,780 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:32,797 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true/_temporary
2014-04-10 11:46:32,811 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:32,831 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:32,832 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/multimerge+true
2014-04-10 11:46:32,850 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:32,850 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:32,860 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:32,861 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:32,865 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:32,870 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [second] starting
2014-04-10 11:46:32,870 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [second]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:32,870 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [second]  sink: Hfs["TextDelimited[['char', 'count']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount"]
2014-04-10 11:46:32,870 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [second]  parallel execution is enabled: false
2014-04-10 11:46:32,870 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [second]  starting jobs: 2
2014-04-10 11:46:32,870 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [second]  allocating threads: 1
2014-04-10 11:46:32,871 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] starting step: (1/2)
2014-04-10 11:46:32,872 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:32,873 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:32,886 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:32,896 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:32,906 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local977641366_0019
2014-04-10 11:46:32,921 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock977641366/.staging/job_local977641366_0019/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:32,921 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock977641366/.staging/job_local977641366_0019/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:32,940 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local977641366_0019/job_local977641366_0019.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:32,940 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local977641366_0019/job_local977641366_0019.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:32,941 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:32,941 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] submitted hadoop job: job_local977641366_0019
2014-04-10 11:46:32,941 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:32,941 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] tracking url: http://localhost:8080/
2014-04-10 11:46:32,941 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:32,943 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:32,944 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local977641366_0019_m_000000_0
2014-04-10 11:46:32,944 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:32,945 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:32,945 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6dcbe1cd
2014-04-10 11:46:32,945 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:32,946 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:32,957 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:32,957 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:32,957 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:32,957 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:32,957 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:32,958 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:32,958 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:32,962 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:32,962 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(first)[by:[{1}:'char']]
2014-04-10 11:46:32,963 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:32,964 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:32,965 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 213, total: 463, max: 672
2014-04-10 11:46:32,965 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:32,965 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:32,965 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 213, total: 463, max: 672
2014-04-10 11:46:32,965 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:32,966 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-04-10 11:46:32,966 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 213, total: 463, max: 672
2014-04-10 11:46:32,966 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:32,966 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-04-10 11:46:32,966 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 213, total: 463, max: 672
2014-04-10 11:46:32,966 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:32,967 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:32,967 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:32,967 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:32,967 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2014-04-10 11:46:32,967 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2014-04-10 11:46:32,969 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:32,969 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local977641366_0019_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:32,970 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:32,970 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local977641366_0019_m_000000_0' done.
2014-04-10 11:46:32,970 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local977641366_0019_m_000000_0
2014-04-10 11:46:32,971 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:32,971 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:32,971 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local977641366_0019_r_000000_0
2014-04-10 11:46:32,972 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:32,972 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@175a9e02
2014-04-10 11:46:32,972 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:32,972 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local977641366_0019_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:32,973 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#15 about to shuffle output of map attempt_local977641366_0019_m_000000_0 decomp: 119 len: 123 to MEMORY
2014-04-10 11:46:32,973 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local977641366_0019_m_000000_0
2014-04-10 11:46:32,973 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2014-04-10 11:46:32,974 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:32,974 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:32,974 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:32,975 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:32,975 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2014-04-10 11:46:32,976 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:32,976 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2014-04-10 11:46:32,976 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:32,976 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:32,976 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2014-04-10 11:46:32,976 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:32,977 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:32,977 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:32,981 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(first)[by:[{1}:'char']]
2014-04-10 11:46:32,981 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][7967949574/first/]
2014-04-10 11:46:32,984 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local977641366_0019_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:32,985 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:32,985 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local977641366_0019_r_000000_0 is allowed to commit now
2014-04-10 11:46:32,985 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local977641366_0019_r_000000_0' to file:/tmp/hadoop-brock/7967949574_first_271D61101AE74C8A8560963E2E64D06E/_temporary/0/task_local977641366_0019_r_000000
2014-04-10 11:46:32,986 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:32,986 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local977641366_0019_r_000000_0' done.
2014-04-10 11:46:32,986 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local977641366_0019_r_000000_0
2014-04-10 11:46:32,986 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:32,995 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] starting step: (2/2) ...form/samesourcemergecount
2014-04-10 11:46:32,995 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:32,996 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:33,006 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:33,016 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:33,022 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:33,022 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:33,027 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1635259541_0020
2014-04-10 11:46:33,036 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1635259541/.staging/job_local1635259541_0020/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:33,036 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1635259541/.staging/job_local1635259541_0020/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:33,056 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1635259541_0020/job_local1635259541_0020.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:33,057 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1635259541_0020/job_local1635259541_0020.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:33,057 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:33,057 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] submitted hadoop job: job_local1635259541_0020
2014-04-10 11:46:33,057 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:33,057 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] tracking url: http://localhost:8080/
2014-04-10 11:46:33,057 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:33,060 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:33,060 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1635259541_0020_m_000000_0
2014-04-10 11:46:33,061 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:33,062 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-brock/7967949574_first_271D61101AE74C8A8560963E2E64D06E/part-00000
2014-04-10 11:46:33,062 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3bd5f99b
2014-04-10 11:46:33,064 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:33,064 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:33,080 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:33,080 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:33,081 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:33,081 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:33,081 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:33,082 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:33,082 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:33,086 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][7967949574/first/]
2014-04-10 11:46:33,086 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(second)[by:[{1}:'char']]
2014-04-10 11:46:33,086 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:33,087 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:33,087 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 249, total: 463, max: 672
2014-04-10 11:46:33,087 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:33,087 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:33,087 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 249, total: 463, max: 672
2014-04-10 11:46:33,088 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:33,088 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:33,088 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:33,088 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:33,088 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 55; bufvoid = 104857600
2014-04-10 11:46:33,089 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2014-04-10 11:46:33,090 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:33,090 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1635259541_0020_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:33,091 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-brock/7967949574_first_271D61101AE74C8A8560963E2E64D06E/part-00000:0+160
2014-04-10 11:46:33,091 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1635259541_0020_m_000000_0' done.
2014-04-10 11:46:33,091 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1635259541_0020_m_000000_0
2014-04-10 11:46:33,092 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:33,092 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:33,092 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1635259541_0020_r_000000_0
2014-04-10 11:46:33,093 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:33,093 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@c4ff005
2014-04-10 11:46:33,093 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:33,094 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1635259541_0020_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:33,095 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#16 about to shuffle output of map attempt_local1635259541_0020_m_000000_0 decomp: 67 len: 71 to MEMORY
2014-04-10 11:46:33,095 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 67 bytes from map-output for attempt_local1635259541_0020_m_000000_0
2014-04-10 11:46:33,095 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 67, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->67
2014-04-10 11:46:33,096 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:33,097 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:33,097 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:33,097 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:33,098 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2014-04-10 11:46:33,098 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 67 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:33,098 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 71 bytes from disk
2014-04-10 11:46:33,098 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:33,099 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:33,099 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2014-04-10 11:46:33,099 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:33,100 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:33,100 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:33,105 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(second)[by:[{1}:'char']]
2014-04-10 11:46:33,105 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount"]
2014-04-10 11:46:33,110 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1635259541_0020_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:33,111 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:33,111 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1635259541_0020_r_000000_0 is allowed to commit now
2014-04-10 11:46:33,111 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1635259541_0020_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount/_temporary/0/task_local1635259541_0020_r_000000
2014-04-10 11:46:33,112 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:33,112 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1635259541_0020_r_000000_0' done.
2014-04-10 11:46:33,112 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1635259541_0020_r_000000_0
2014-04-10 11:46:33,112 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:33,123 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount/_temporary
2014-04-10 11:46:33,136 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:33,147 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:33,147 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount
2014-04-10 11:46:33,156 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:33,156 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:33,159 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:33,160 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:33,160 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:33,171 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [average] starting
2014-04-10 11:46:33,171 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [average]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:33,171 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [average]  sink: Hfs["TextDelimited[['char', 'average']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average"]
2014-04-10 11:46:33,171 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [average]  parallel execution is enabled: false
2014-04-10 11:46:33,171 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [average]  starting jobs: 1
2014-04-10 11:46:33,171 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [average]  allocating threads: 1
2014-04-10 11:46:33,171 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] starting step: (1/1) ...lyhelpersplatform/average
2014-04-10 11:46:33,172 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:33,172 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:33,179 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:33,184 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:33,189 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1058851983_0021
2014-04-10 11:46:33,196 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1058851983/.staging/job_local1058851983_0021/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:33,196 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1058851983/.staging/job_local1058851983_0021/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:33,216 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1058851983_0021/job_local1058851983_0021.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:33,217 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1058851983_0021/job_local1058851983_0021.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:33,217 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:33,217 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] submitted hadoop job: job_local1058851983_0021
2014-04-10 11:46:33,217 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:33,217 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] tracking url: http://localhost:8080/
2014-04-10 11:46:33,218 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:33,219 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:33,219 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1058851983_0021_m_000000_0
2014-04-10 11:46:33,219 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:33,220 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:33,220 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7d27a2b6
2014-04-10 11:46:33,220 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:33,221 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:33,233 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:33,234 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:33,234 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:33,234 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:33,234 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:33,234 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:33,235 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:33,238 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:33,238 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(average)[by:[{1}:'char']]
2014-04-10 11:46:33,238 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:33,239 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:33,239 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 252, total: 465, max: 672
2014-04-10 11:46:33,239 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:33,239 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:33,239 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 252, total: 465, max: 672
2014-04-10 11:46:33,239 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:33,240 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-04-10 11:46:33,240 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 252, total: 465, max: 672
2014-04-10 11:46:33,240 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:33,240 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-04-10 11:46:33,240 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 252, total: 465, max: 672
2014-04-10 11:46:33,240 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:33,241 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:33,241 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:33,241 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:33,241 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2014-04-10 11:46:33,241 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2014-04-10 11:46:33,242 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:33,243 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1058851983_0021_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:33,244 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:33,244 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1058851983_0021_m_000000_0' done.
2014-04-10 11:46:33,244 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1058851983_0021_m_000000_0
2014-04-10 11:46:33,244 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:33,244 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:33,245 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1058851983_0021_r_000000_0
2014-04-10 11:46:33,245 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:33,245 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@48b98610
2014-04-10 11:46:33,246 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:33,246 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1058851983_0021_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:33,247 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#17 about to shuffle output of map attempt_local1058851983_0021_m_000000_0 decomp: 200 len: 204 to MEMORY
2014-04-10 11:46:33,248 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local1058851983_0021_m_000000_0
2014-04-10 11:46:33,248 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->200
2014-04-10 11:46:33,248 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:33,249 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:33,249 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:33,251 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:33,251 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2014-04-10 11:46:33,252 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 200 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:33,252 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 204 bytes from disk
2014-04-10 11:46:33,253 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:33,253 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:33,253 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2014-04-10 11:46:33,254 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:33,254 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:33,254 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:33,260 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(average)[by:[{1}:'char']]
2014-04-10 11:46:33,260 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average"]
2014-04-10 11:46:33,265 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1058851983_0021_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:33,267 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:33,267 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1058851983_0021_r_000000_0 is allowed to commit now
2014-04-10 11:46:33,267 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1058851983_0021_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average/_temporary/0/task_local1058851983_0021_r_000000
2014-04-10 11:46:33,268 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:33,268 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1058851983_0021_r_000000_0' done.
2014-04-10 11:46:33,268 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1058851983_0021_r_000000_0
2014-04-10 11:46:33,268 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:33,282 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average/_temporary
2014-04-10 11:46:33,303 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:33,325 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:33,326 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/average
2014-04-10 11:46:33,342 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:33,342 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:33,347 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:33,348 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:33,348 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:33,354 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [sum] starting
2014-04-10 11:46:33,354 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [sum]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:33,354 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [sum]  sink: Hfs["TextDelimited[['char', 'sum']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls"]
2014-04-10 11:46:33,354 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [sum]  parallel execution is enabled: false
2014-04-10 11:46:33,354 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [sum]  starting jobs: 1
2014-04-10 11:46:33,354 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [sum]  allocating threads: 1
2014-04-10 11:46:33,354 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] starting step: (1/1) ...yhelpersplatform/sumnulls
2014-04-10 11:46:33,355 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:33,355 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:33,362 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:33,369 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:33,380 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local572349713_0022
2014-04-10 11:46:33,392 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock572349713/.staging/job_local572349713_0022/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:33,392 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock572349713/.staging/job_local572349713_0022/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:33,411 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local572349713_0022/job_local572349713_0022.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:33,411 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local572349713_0022/job_local572349713_0022.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:33,412 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:33,412 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] submitted hadoop job: job_local572349713_0022
2014-04-10 11:46:33,412 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:33,412 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] tracking url: http://localhost:8080/
2014-04-10 11:46:33,412 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:33,415 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:33,415 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local572349713_0022_m_000000_0
2014-04-10 11:46:33,416 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:33,417 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:33,417 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@c3e45b9
2014-04-10 11:46:33,418 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:33,418 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:33,433 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:33,433 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:33,433 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:33,433 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:33,433 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:33,434 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:33,434 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:33,437 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:33,437 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(sum)[by:[{1}:'char']]
2014-04-10 11:46:33,437 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:33,440 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:33,440 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 251, total: 464, max: 672
2014-04-10 11:46:33,440 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:33,441 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:33,441 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 251, total: 464, max: 672
2014-04-10 11:46:33,441 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:33,441 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-04-10 11:46:33,441 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 251, total: 464, max: 672
2014-04-10 11:46:33,441 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:33,442 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-04-10 11:46:33,442 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 251, total: 464, max: 672
2014-04-10 11:46:33,442 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:33,442 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:33,442 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:33,442 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:33,442 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 96; bufvoid = 104857600
2014-04-10 11:46:33,443 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2014-04-10 11:46:33,444 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:33,444 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local572349713_0022_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:33,445 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:33,445 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local572349713_0022_m_000000_0' done.
2014-04-10 11:46:33,445 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local572349713_0022_m_000000_0
2014-04-10 11:46:33,445 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:33,446 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:33,446 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local572349713_0022_r_000000_0
2014-04-10 11:46:33,447 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:33,448 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@1af70aa3
2014-04-10 11:46:33,448 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:33,449 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local572349713_0022_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:33,450 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#18 about to shuffle output of map attempt_local572349713_0022_m_000000_0 decomp: 116 len: 120 to MEMORY
2014-04-10 11:46:33,451 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 116 bytes from map-output for attempt_local572349713_0022_m_000000_0
2014-04-10 11:46:33,451 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 116, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->116
2014-04-10 11:46:33,452 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:33,452 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:33,453 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:33,454 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:33,454 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 107 bytes
2014-04-10 11:46:33,455 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 116 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:33,456 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 120 bytes from disk
2014-04-10 11:46:33,456 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:33,456 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:33,456 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 107 bytes
2014-04-10 11:46:33,457 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:33,457 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:33,457 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:33,465 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(sum)[by:[{1}:'char']]
2014-04-10 11:46:33,465 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls"]
2014-04-10 11:46:33,472 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local572349713_0022_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:33,474 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:33,474 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local572349713_0022_r_000000_0 is allowed to commit now
2014-04-10 11:46:33,475 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local572349713_0022_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls/_temporary/0/task_local572349713_0022_r_000000
2014-04-10 11:46:33,476 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:33,476 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local572349713_0022_r_000000_0' done.
2014-04-10 11:46:33,476 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local572349713_0022_r_000000_0
2014-04-10 11:46:33,477 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:33,489 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls/_temporary
2014-04-10 11:46:33,507 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:33,523 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:33,524 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/sumnulls
2014-04-10 11:46:33,534 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:33,534 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:33,538 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:33,539 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:33,539 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:33,543 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count] starting
2014-04-10 11:46:33,543 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:33,543 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count]  sink: Hfs["TextDelimited[['count']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall"]
2014-04-10 11:46:33,543 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count]  parallel execution is enabled: false
2014-04-10 11:46:33,543 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count]  starting jobs: 1
2014-04-10 11:46:33,544 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count]  allocating threads: 1
2014-04-10 11:46:33,544 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...yhelpersplatform/countall
2014-04-10 11:46:33,544 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:33,545 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:33,558 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:33,567 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:33,577 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local2082772672_0023
2014-04-10 11:46:33,591 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock2082772672/.staging/job_local2082772672_0023/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:33,591 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock2082772672/.staging/job_local2082772672_0023/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:33,610 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local2082772672_0023/job_local2082772672_0023.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:33,610 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local2082772672_0023/job_local2082772672_0023.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:33,610 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:33,611 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local2082772672_0023
2014-04-10 11:46:33,611 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:33,611 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:33,611 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2014-04-10 11:46:33,613 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:33,613 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local2082772672_0023_m_000000_0
2014-04-10 11:46:33,615 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:33,616 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:33,616 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4d95dd74
2014-04-10 11:46:33,617 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:33,617 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:33,634 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:33,635 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:33,635 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:33,635 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:33,635 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:33,636 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:33,636 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:33,641 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:33,641 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(count)[by:[{?}:NONE]]
2014-04-10 11:46:33,641 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:33,642 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:33,642 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:33,642 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:33,642 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 4; bufvoid = 104857600
2014-04-10 11:46:33,643 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214396(104857584); length = 1/6553600
2014-04-10 11:46:33,643 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:33,644 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local2082772672_0023_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:33,645 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:33,645 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2082772672_0023_m_000000_0' done.
2014-04-10 11:46:33,645 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local2082772672_0023_m_000000_0
2014-04-10 11:46:33,645 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:33,646 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:33,646 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local2082772672_0023_r_000000_0
2014-04-10 11:46:33,646 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:33,647 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@3ff800b7
2014-04-10 11:46:33,647 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:33,647 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local2082772672_0023_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:33,648 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#19 about to shuffle output of map attempt_local2082772672_0023_m_000000_0 decomp: 8 len: 12 to MEMORY
2014-04-10 11:46:33,649 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 8 bytes from map-output for attempt_local2082772672_0023_m_000000_0
2014-04-10 11:46:33,649 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 8, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->8
2014-04-10 11:46:33,650 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:33,650 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:33,650 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:33,651 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:33,651 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 5 bytes
2014-04-10 11:46:33,652 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 8 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:33,652 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 12 bytes from disk
2014-04-10 11:46:33,652 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:33,652 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:33,652 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 5 bytes
2014-04-10 11:46:33,652 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:33,652 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:33,653 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:33,656 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(count)[by:[{?}:NONE]]
2014-04-10 11:46:33,656 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['count']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall"]
2014-04-10 11:46:33,660 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local2082772672_0023_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:33,662 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:33,662 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local2082772672_0023_r_000000_0 is allowed to commit now
2014-04-10 11:46:33,663 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local2082772672_0023_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall/_temporary/0/task_local2082772672_0023_r_000000
2014-04-10 11:46:33,664 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:33,664 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2082772672_0023_r_000000_0' done.
2014-04-10 11:46:33,664 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local2082772672_0023_r_000000_0
2014-04-10 11:46:33,665 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:33,677 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall/_temporary
2014-04-10 11:46:33,694 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:33,714 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:33,716 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/countall
2014-04-10 11:46:33,731 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:33,732 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:33,735 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:33,736 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:33,737 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:33,741 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [name] starting
2014-04-10 11:46:33,741 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [name]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:33,741 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [name]  sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average', 'average2', 'first']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi"]
2014-04-10 11:46:33,742 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [name]  parallel execution is enabled: false
2014-04-10 11:46:33,742 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [name]  starting jobs: 1
2014-04-10 11:46:33,742 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [name]  allocating threads: 1
2014-04-10 11:46:33,742 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] starting step: (1/1) ...mblyhelpersplatform/multi
2014-04-10 11:46:33,743 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:33,743 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:33,752 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:33,757 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:33,762 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local297220638_0024
2014-04-10 11:46:33,769 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock297220638/.staging/job_local297220638_0024/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:33,770 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock297220638/.staging/job_local297220638_0024/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:33,789 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local297220638_0024/job_local297220638_0024.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:33,789 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local297220638_0024/job_local297220638_0024.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:33,790 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:33,790 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] submitted hadoop job: job_local297220638_0024
2014-04-10 11:46:33,790 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] tracking url: http://localhost:8080/
2014-04-10 11:46:33,790 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:33,790 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:33,792 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:33,792 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local297220638_0024_m_000000_0
2014-04-10 11:46:33,793 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:33,793 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:33,793 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@32c6a74b
2014-04-10 11:46:33,794 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:33,794 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:33,807 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:33,808 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:33,808 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:33,808 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:33,808 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:33,809 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:33,809 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:33,813 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:33,813 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2014-04-10 11:46:33,813 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:33,814 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:33,814 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 253, total: 466, max: 672
2014-04-10 11:46:33,814 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:33,814 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:33,815 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 253, total: 466, max: 672
2014-04-10 11:46:33,815 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:33,815 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-04-10 11:46:33,815 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 253, total: 466, max: 672
2014-04-10 11:46:33,815 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:33,816 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-04-10 11:46:33,816 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 253, total: 466, max: 672
2014-04-10 11:46:33,816 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:33,816 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:33,817 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:33,817 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:33,817 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 432; bufvoid = 104857600
2014-04-10 11:46:33,817 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2014-04-10 11:46:33,818 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:33,819 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local297220638_0024_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:33,820 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:33,820 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local297220638_0024_m_000000_0' done.
2014-04-10 11:46:33,820 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local297220638_0024_m_000000_0
2014-04-10 11:46:33,820 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:33,820 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:33,820 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local297220638_0024_r_000000_0
2014-04-10 11:46:33,822 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:33,822 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@30122f83
2014-04-10 11:46:33,823 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:33,823 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local297220638_0024_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:33,824 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#20 about to shuffle output of map attempt_local297220638_0024_m_000000_0 decomp: 452 len: 456 to MEMORY
2014-04-10 11:46:33,825 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 452 bytes from map-output for attempt_local297220638_0024_m_000000_0
2014-04-10 11:46:33,825 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 452, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->452
2014-04-10 11:46:33,825 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:33,826 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:33,826 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:33,827 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:33,827 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 436 bytes
2014-04-10 11:46:33,828 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 452 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:33,828 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 456 bytes from disk
2014-04-10 11:46:33,828 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:33,828 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:33,828 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 436 bytes
2014-04-10 11:46:33,829 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:33,829 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:33,829 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:33,833 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(name)[by:[{1}:'char']]
2014-04-10 11:46:33,833 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average', 'average2', 'first']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi"]
2014-04-10 11:46:33,840 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local297220638_0024_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:33,842 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:33,842 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local297220638_0024_r_000000_0 is allowed to commit now
2014-04-10 11:46:33,843 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local297220638_0024_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi/_temporary/0/task_local297220638_0024_r_000000
2014-04-10 11:46:33,844 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:33,844 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local297220638_0024_r_000000_0' done.
2014-04-10 11:46:33,844 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local297220638_0024_r_000000_0
2014-04-10 11:46:33,845 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:33,857 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi/_temporary
2014-04-10 11:46:33,878 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:33,896 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:33,897 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/multi
2014-04-10 11:46:33,908 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:33,909 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:33,911 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:33,911 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:33,912 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:33,915 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] starting
2014-04-10 11:46:33,915 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt"]
2014-04-10 11:46:33,915 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  sink: Hfs["TextLine[['num']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow"]
2014-04-10 11:46:33,915 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  parallel execution is enabled: false
2014-04-10 11:46:33,915 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  starting jobs: 1
2014-04-10 11:46:33,915 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  allocating threads: 1
2014-04-10 11:46:33,916 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...persplatform/retainnarrow
2014-04-10 11:46:33,916 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:33,917 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:33,930 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:33,941 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:33,952 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1462269840_0025
2014-04-10 11:46:33,962 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1462269840/.staging/job_local1462269840_0025/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:33,963 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1462269840/.staging/job_local1462269840_0025/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:33,984 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1462269840_0025/job_local1462269840_0025.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:33,984 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1462269840_0025/job_local1462269840_0025.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:33,985 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:33,985 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1462269840_0025
2014-04-10 11:46:33,985 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2014-04-10 11:46:33,985 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:33,985 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:33,987 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:33,987 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1462269840_0025_m_000000_0
2014-04-10 11:46:33,989 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:33,989 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt
2014-04-10 11:46:33,989 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@56d5c758
2014-04-10 11:46:33,990 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2014-04-10 11:46:33,993 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:33,993 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:33,998 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt"]
2014-04-10 11:46:33,998 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: Hfs["TextLine[['num']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow"]
2014-04-10 11:46:34,000 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:34,000 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1462269840_0025_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:34,000 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:34,000 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1462269840_0025_m_000000_0 is allowed to commit now
2014-04-10 11:46:34,001 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1462269840_0025_m_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow/_temporary/0/task_local1462269840_0025_m_000000
2014-04-10 11:46:34,001 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt:0+19
2014-04-10 11:46:34,001 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1462269840_0025_m_000000_0' done.
2014-04-10 11:46:34,002 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1462269840_0025_m_000000_0
2014-04-10 11:46:34,002 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:34,007 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow/_temporary
2014-04-10 11:46:34,017 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:34,017 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/retainnarrow
2014-04-10 11:46:34,025 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:34,025 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:34,027 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:34,028 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:34,029 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:34,031 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] starting
2014-04-10 11:46:34,031 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt"]
2014-04-10 11:46:34,031 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  sink: Hfs["TextLine[['item']->['char', 'item']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow"]
2014-04-10 11:46:34,031 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  parallel execution is enabled: false
2014-04-10 11:46:34,031 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  starting jobs: 1
2014-04-10 11:46:34,031 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  allocating threads: 1
2014-04-10 11:46:34,031 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...persplatform/renamenarrow
2014-04-10 11:46:34,032 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:34,032 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:34,039 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:34,044 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:34,048 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local879190019_0026
2014-04-10 11:46:34,059 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock879190019/.staging/job_local879190019_0026/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:34,060 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock879190019/.staging/job_local879190019_0026/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:34,084 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local879190019_0026/job_local879190019_0026.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:34,084 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local879190019_0026/job_local879190019_0026.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:34,085 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:34,085 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local879190019_0026
2014-04-10 11:46:34,085 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2014-04-10 11:46:34,085 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:34,085 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:34,087 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:34,087 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local879190019_0026_m_000000_0
2014-04-10 11:46:34,089 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:34,090 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt
2014-04-10 11:46:34,090 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3486f312
2014-04-10 11:46:34,091 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2014-04-10 11:46:34,095 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:34,095 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:34,098 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt"]
2014-04-10 11:46:34,098 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: Hfs["TextLine[['item']->['char', 'item']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow"]
2014-04-10 11:46:34,100 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:34,100 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local879190019_0026_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:34,102 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:34,102 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local879190019_0026_m_000000_0 is allowed to commit now
2014-04-10 11:46:34,103 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local879190019_0026_m_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow/_temporary/0/task_local879190019_0026_m_000000
2014-04-10 11:46:34,104 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt:0+19
2014-04-10 11:46:34,104 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local879190019_0026_m_000000_0' done.
2014-04-10 11:46:34,104 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local879190019_0026_m_000000_0
2014-04-10 11:46:34,105 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:34,119 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow/_temporary
2014-04-10 11:46:34,135 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:34,136 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/renamenarrow
2014-04-10 11:46:34,145 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:34,145 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:34,148 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:34,149 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:34,149 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:34,152 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [average] starting
2014-04-10 11:46:34,153 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [average]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:34,153 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [average]  sink: Hfs["TextDelimited[['char', 'average']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull"]
2014-04-10 11:46:34,153 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [average]  parallel execution is enabled: false
2014-04-10 11:46:34,153 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [average]  starting jobs: 1
2014-04-10 11:46:34,153 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [average]  allocating threads: 1
2014-04-10 11:46:34,153 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] starting step: (1/1) ...lpersplatform/averagenull
2014-04-10 11:46:34,153 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:34,154 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:34,160 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:34,165 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:34,172 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local916075696_0027
2014-04-10 11:46:34,182 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock916075696/.staging/job_local916075696_0027/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:34,182 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock916075696/.staging/job_local916075696_0027/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:34,213 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local916075696_0027/job_local916075696_0027.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:34,213 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local916075696_0027/job_local916075696_0027.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:34,214 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:34,214 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] submitted hadoop job: job_local916075696_0027
2014-04-10 11:46:34,214 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:34,214 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] tracking url: http://localhost:8080/
2014-04-10 11:46:34,214 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:34,216 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:34,216 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local916075696_0027_m_000000_0
2014-04-10 11:46:34,218 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:34,219 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:34,219 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4ea2d2b0
2014-04-10 11:46:34,220 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:34,220 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:34,233 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:34,233 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:34,233 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:34,234 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:34,234 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:34,234 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:34,234 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:34,240 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:34,240 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(average)[by:[{1}:'char']]
2014-04-10 11:46:34,240 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:34,242 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:34,242 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 241, total: 467, max: 672
2014-04-10 11:46:34,242 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:34,243 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:34,243 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 241, total: 467, max: 672
2014-04-10 11:46:34,243 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:34,243 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-04-10 11:46:34,243 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 241, total: 467, max: 672
2014-04-10 11:46:34,243 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:34,244 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-04-10 11:46:34,244 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 240, total: 467, max: 672
2014-04-10 11:46:34,244 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:34,245 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:34,245 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:34,245 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:34,245 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2014-04-10 11:46:34,245 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2014-04-10 11:46:34,246 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:34,247 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local916075696_0027_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:34,248 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:34,248 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local916075696_0027_m_000000_0' done.
2014-04-10 11:46:34,248 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local916075696_0027_m_000000_0
2014-04-10 11:46:34,248 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:34,248 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:34,248 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local916075696_0027_r_000000_0
2014-04-10 11:46:34,249 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:34,249 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@6f94a464
2014-04-10 11:46:34,249 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:34,250 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local916075696_0027_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:34,250 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#21 about to shuffle output of map attempt_local916075696_0027_m_000000_0 decomp: 200 len: 204 to MEMORY
2014-04-10 11:46:34,250 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local916075696_0027_m_000000_0
2014-04-10 11:46:34,251 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->200
2014-04-10 11:46:34,251 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:34,251 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:34,251 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:34,252 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:34,252 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2014-04-10 11:46:34,253 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 200 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:34,253 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 204 bytes from disk
2014-04-10 11:46:34,253 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:34,253 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:34,254 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2014-04-10 11:46:34,254 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:34,254 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:34,254 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:34,257 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(average)[by:[{1}:'char']]
2014-04-10 11:46:34,258 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull"]
2014-04-10 11:46:34,263 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local916075696_0027_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:34,265 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:34,265 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local916075696_0027_r_000000_0 is allowed to commit now
2014-04-10 11:46:34,266 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local916075696_0027_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull/_temporary/0/task_local916075696_0027_r_000000
2014-04-10 11:46:34,267 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:34,267 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local916075696_0027_r_000000_0' done.
2014-04-10 11:46:34,267 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local916075696_0027_r_000000_0
2014-04-10 11:46:34,267 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:34,281 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull/_temporary
2014-04-10 11:46:34,297 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:34,316 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:34,318 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/averagenull
2014-04-10 11:46:34,330 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:34,331 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:34,333 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:34,333 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:34,334 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:34,336 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce] starting
2014-04-10 11:46:34,336 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt"]
2014-04-10 11:46:34,336 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce]  sink: Hfs["TextLine[['line']->['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce"]
2014-04-10 11:46:34,337 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce]  parallel execution is enabled: false
2014-04-10 11:46:34,337 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce]  starting jobs: 1
2014-04-10 11:46:34,337 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce]  allocating threads: 1
2014-04-10 11:46:34,337 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] starting step: (1/1) ...blyhelpersplatform/coerce
2014-04-10 11:46:34,338 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:34,339 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:34,350 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:34,360 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:34,370 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local764195239_0028
2014-04-10 11:46:34,381 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock764195239/.staging/job_local764195239_0028/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:34,382 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock764195239/.staging/job_local764195239_0028/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:34,411 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local764195239_0028/job_local764195239_0028.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:34,411 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local764195239_0028/job_local764195239_0028.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:34,411 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:34,412 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] submitted hadoop job: job_local764195239_0028
2014-04-10 11:46:34,412 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] tracking url: http://localhost:8080/
2014-04-10 11:46:34,412 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:34,412 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:34,414 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:34,414 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local764195239_0028_m_000000_0
2014-04-10 11:46:34,415 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:34,416 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt
2014-04-10 11:46:34,417 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@d862c40
2014-04-10 11:46:34,418 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2014-04-10 11:46:34,422 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:34,422 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:34,427 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt"]
2014-04-10 11:46:34,427 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: Hfs["TextLine[['line']->['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce"]
2014-04-10 11:46:34,429 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:34,429 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local764195239_0028_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:34,430 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:34,430 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local764195239_0028_m_000000_0 is allowed to commit now
2014-04-10 11:46:34,430 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local764195239_0028_m_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce/_temporary/0/task_local764195239_0028_m_000000
2014-04-10 11:46:34,431 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt:0+19
2014-04-10 11:46:34,431 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local764195239_0028_m_000000_0' done.
2014-04-10 11:46:34,431 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local764195239_0028_m_000000_0
2014-04-10 11:46:34,431 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:34,443 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce/_temporary
2014-04-10 11:46:34,457 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:34,458 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/coerce
2014-04-10 11:46:34,472 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:34,473 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:34,486 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:34,487 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:34,499 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:34,502 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs*source*count...] starting
2014-04-10 11:46:34,503 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs*source*count...]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:34,503 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs*source*count...]  sink: Hfs["TextDelimited[['char', 'count']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway"]
2014-04-10 11:46:34,503 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs*source*count...]  parallel execution is enabled: false
2014-04-10 11:46:34,503 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs*source*count...]  starting jobs: 5
2014-04-10 11:46:34,503 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs*source*count...]  allocating threads: 1
2014-04-10 11:46:34,503 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (1/5)
2014-04-10 11:46:34,504 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:34,504 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:34,517 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:34,526 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:34,536 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local652454393_0029
2014-04-10 11:46:34,550 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock652454393/.staging/job_local652454393_0029/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:34,550 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock652454393/.staging/job_local652454393_0029/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:34,574 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local652454393_0029/job_local652454393_0029.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:34,574 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local652454393_0029/job_local652454393_0029.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:34,574 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:34,575 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local652454393_0029
2014-04-10 11:46:34,575 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:34,575 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2014-04-10 11:46:34,575 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:34,576 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:34,577 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local652454393_0029_m_000000_0
2014-04-10 11:46:34,578 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:34,579 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:34,579 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2aff12fc
2014-04-10 11:46:34,579 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:34,579 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:34,593 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:34,593 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:34,593 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:34,593 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:34,593 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:34,594 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:34,594 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:34,598 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:34,598 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(source)[by:[{1}:'char']]
2014-04-10 11:46:34,598 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:34,599 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:34,599 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 252, total: 467, max: 672
2014-04-10 11:46:34,599 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:34,599 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:34,599 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 252, total: 467, max: 672
2014-04-10 11:46:34,599 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:34,600 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-04-10 11:46:34,600 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 252, total: 467, max: 672
2014-04-10 11:46:34,600 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:34,600 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-04-10 11:46:34,600 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 252, total: 467, max: 672
2014-04-10 11:46:34,600 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:34,601 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:34,601 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:34,601 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:34,601 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2014-04-10 11:46:34,601 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2014-04-10 11:46:34,602 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:34,603 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local652454393_0029_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:34,603 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:34,603 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local652454393_0029_m_000000_0' done.
2014-04-10 11:46:34,603 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local652454393_0029_m_000000_0
2014-04-10 11:46:34,604 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:34,604 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:34,604 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local652454393_0029_r_000000_0
2014-04-10 11:46:34,606 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:34,606 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@4f26fac4
2014-04-10 11:46:34,606 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:34,607 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local652454393_0029_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:34,608 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#22 about to shuffle output of map attempt_local652454393_0029_m_000000_0 decomp: 119 len: 123 to MEMORY
2014-04-10 11:46:34,609 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local652454393_0029_m_000000_0
2014-04-10 11:46:34,609 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2014-04-10 11:46:34,610 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:34,610 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:34,611 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:34,612 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:34,612 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2014-04-10 11:46:34,614 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:34,614 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2014-04-10 11:46:34,614 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:34,614 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:34,615 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2014-04-10 11:46:34,616 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:34,616 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:34,616 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:34,622 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(source)[by:[{1}:'char']]
2014-04-10 11:46:34,622 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][4315789487/source/]
2014-04-10 11:46:34,628 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local652454393_0029_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:34,629 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:34,629 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local652454393_0029_r_000000_0 is allowed to commit now
2014-04-10 11:46:34,630 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local652454393_0029_r_000000_0' to file:/tmp/hadoop-brock/4315789487_source_27ABE25713A0429A9EB8DCA7E343CE75/_temporary/0/task_local652454393_0029_r_000000
2014-04-10 11:46:34,630 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:34,630 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local652454393_0029_r_000000_0' done.
2014-04-10 11:46:34,630 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local652454393_0029_r_000000_0
2014-04-10 11:46:34,630 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:34,639 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (3/5)
2014-04-10 11:46:34,640 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:34,640 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:34,654 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:34,658 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:34,661 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:34,661 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:34,663 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1859977784_0030
2014-04-10 11:46:34,671 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1859977784/.staging/job_local1859977784_0030/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:34,671 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1859977784/.staging/job_local1859977784_0030/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:34,689 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1859977784_0030/job_local1859977784_0030.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:34,690 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1859977784_0030/job_local1859977784_0030.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:34,690 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:34,690 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1859977784_0030
2014-04-10 11:46:34,690 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:34,690 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2014-04-10 11:46:34,691 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:34,692 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:34,692 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1859977784_0030_m_000000_0
2014-04-10 11:46:34,692 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:34,693 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-brock/4315789487_source_27ABE25713A0429A9EB8DCA7E343CE75/part-00000
2014-04-10 11:46:34,693 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5595c0fb
2014-04-10 11:46:34,693 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:34,693 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:34,707 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:34,707 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:34,707 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:34,707 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:34,707 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:34,707 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:34,707 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:34,710 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][4315789487/source/]
2014-04-10 11:46:34,710 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(count-lhs)[by:[{1}:'char']]
2014-04-10 11:46:34,710 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:34,711 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:34,711 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 255, total: 468, max: 672
2014-04-10 11:46:34,711 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:34,711 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:34,711 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 255, total: 468, max: 672
2014-04-10 11:46:34,711 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:34,712 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:34,712 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:34,712 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:34,712 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 55; bufvoid = 104857600
2014-04-10 11:46:34,712 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2014-04-10 11:46:34,713 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:34,714 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1859977784_0030_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:34,714 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-brock/4315789487_source_27ABE25713A0429A9EB8DCA7E343CE75/part-00000:0+160
2014-04-10 11:46:34,714 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1859977784_0030_m_000000_0' done.
2014-04-10 11:46:34,715 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1859977784_0030_m_000000_0
2014-04-10 11:46:34,715 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:34,715 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:34,715 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1859977784_0030_r_000000_0
2014-04-10 11:46:34,716 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:34,716 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@27b68be7
2014-04-10 11:46:34,716 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:34,716 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1859977784_0030_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:34,717 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#23 about to shuffle output of map attempt_local1859977784_0030_m_000000_0 decomp: 67 len: 71 to MEMORY
2014-04-10 11:46:34,717 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 67 bytes from map-output for attempt_local1859977784_0030_m_000000_0
2014-04-10 11:46:34,717 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 67, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->67
2014-04-10 11:46:34,717 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:34,718 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:34,718 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:34,719 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:34,719 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2014-04-10 11:46:34,719 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 67 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:34,719 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 71 bytes from disk
2014-04-10 11:46:34,719 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:34,719 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:34,719 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2014-04-10 11:46:34,720 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:34,720 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:34,720 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:34,722 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(count-lhs)[by:[{1}:'char']]
2014-04-10 11:46:34,723 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][3285150057/count-lhs/]
2014-04-10 11:46:34,725 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1859977784_0030_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:34,726 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:34,726 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1859977784_0030_r_000000_0 is allowed to commit now
2014-04-10 11:46:34,727 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1859977784_0030_r_000000_0' to file:/tmp/hadoop-brock/3285150057_count_lhs_CE5F24697C984271850830A75D0B8D69/_temporary/0/task_local1859977784_0030_r_000000
2014-04-10 11:46:34,727 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:34,727 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1859977784_0030_r_000000_0' done.
2014-04-10 11:46:34,727 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1859977784_0030_r_000000_0
2014-04-10 11:46:34,727 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:34,745 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (4/5)
2014-04-10 11:46:34,746 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:34,747 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:34,756 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:34,761 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:34,764 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:34,764 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:34,766 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1575782110_0031
2014-04-10 11:46:34,773 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1575782110/.staging/job_local1575782110_0031/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:34,773 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1575782110/.staging/job_local1575782110_0031/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:34,792 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1575782110_0031/job_local1575782110_0031.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:34,792 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1575782110_0031/job_local1575782110_0031.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:34,792 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:34,793 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1575782110_0031
2014-04-10 11:46:34,793 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:34,793 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2014-04-10 11:46:34,793 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:34,794 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:34,794 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1575782110_0031_m_000000_0
2014-04-10 11:46:34,794 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:34,795 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-brock/4315789487_source_27ABE25713A0429A9EB8DCA7E343CE75/part-00000
2014-04-10 11:46:34,795 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7429883
2014-04-10 11:46:34,795 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:34,795 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:34,808 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:34,809 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:34,809 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:34,809 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:34,809 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:34,809 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:34,809 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:34,812 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][4315789487/source/]
2014-04-10 11:46:34,812 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(count-rhs)[by:[{1}:'char']]
2014-04-10 11:46:34,812 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:34,813 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-10 11:46:34,813 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 254, total: 468, max: 672
2014-04-10 11:46:34,813 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:34,813 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-10 11:46:34,813 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 254, total: 468, max: 672
2014-04-10 11:46:34,813 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-10 11:46:34,814 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:34,814 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:34,814 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:34,814 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 55; bufvoid = 104857600
2014-04-10 11:46:34,814 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2014-04-10 11:46:34,815 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:34,816 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1575782110_0031_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:34,816 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-brock/4315789487_source_27ABE25713A0429A9EB8DCA7E343CE75/part-00000:0+160
2014-04-10 11:46:34,817 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1575782110_0031_m_000000_0' done.
2014-04-10 11:46:34,817 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1575782110_0031_m_000000_0
2014-04-10 11:46:34,817 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:34,817 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:34,817 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1575782110_0031_r_000000_0
2014-04-10 11:46:34,818 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:34,818 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@2c013c6a
2014-04-10 11:46:34,818 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:34,818 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1575782110_0031_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:34,819 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#24 about to shuffle output of map attempt_local1575782110_0031_m_000000_0 decomp: 67 len: 71 to MEMORY
2014-04-10 11:46:34,819 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 67 bytes from map-output for attempt_local1575782110_0031_m_000000_0
2014-04-10 11:46:34,819 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 67, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->67
2014-04-10 11:46:34,819 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:34,820 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:34,820 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:34,821 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:34,821 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2014-04-10 11:46:34,821 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 67 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:34,821 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 71 bytes from disk
2014-04-10 11:46:34,821 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:34,821 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:34,821 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2014-04-10 11:46:34,822 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:34,822 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:34,822 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:34,824 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(count-rhs)[by:[{1}:'char']]
2014-04-10 11:46:34,824 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][5069137196/count-rhs/]
2014-04-10 11:46:34,827 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1575782110_0031_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:34,828 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:34,828 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1575782110_0031_r_000000_0 is allowed to commit now
2014-04-10 11:46:34,829 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1575782110_0031_r_000000_0' to file:/tmp/hadoop-brock/5069137196_count_rhs_EDA8141FE1BA4053A420E3FE3FBD41E2/_temporary/0/task_local1575782110_0031_r_000000
2014-04-10 11:46:34,829 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:34,829 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1575782110_0031_r_000000_0' done.
2014-04-10 11:46:34,830 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1575782110_0031_r_000000_0
2014-04-10 11:46:34,830 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:34,847 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (2/5)
2014-04-10 11:46:34,848 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:34,848 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:34,865 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:34,872 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:34,881 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2014-04-10 11:46:34,886 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:34,886 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:34,891 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1285614220_0032
2014-04-10 11:46:34,902 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1285614220/.staging/job_local1285614220_0032/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:34,903 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1285614220/.staging/job_local1285614220_0032/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:34,931 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1285614220_0032/job_local1285614220_0032.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:34,931 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1285614220_0032/job_local1285614220_0032.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:34,931 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:34,931 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1285614220_0032
2014-04-10 11:46:34,932 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:34,932 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2014-04-10 11:46:34,932 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:34,934 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:34,934 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1285614220_0032_m_000000_0
2014-04-10 11:46:34,935 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:34,936 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-brock/5069137196_count_rhs_EDA8141FE1BA4053A420E3FE3FBD41E2/part-00000
2014-04-10 11:46:34,937 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@46f7adc4
2014-04-10 11:46:34,938 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:34,938 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:34,953 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:34,953 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:34,953 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:34,953 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:34,953 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:34,955 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:34,955 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:34,958 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][5069137196/count-rhs/]
2014-04-10 11:46:34,958 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2014-04-10 11:46:34,960 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:34,960 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:34,960 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:34,960 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 65; bufvoid = 104857600
2014-04-10 11:46:34,960 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2014-04-10 11:46:34,962 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:34,964 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1285614220_0032_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:34,966 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-brock/5069137196_count_rhs_EDA8141FE1BA4053A420E3FE3FBD41E2/part-00000:0+160
2014-04-10 11:46:34,966 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1285614220_0032_m_000000_0' done.
2014-04-10 11:46:34,966 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1285614220_0032_m_000000_0
2014-04-10 11:46:34,966 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1285614220_0032_m_000001_0
2014-04-10 11:46:34,967 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:34,968 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-brock/4315789487_source_27ABE25713A0429A9EB8DCA7E343CE75/part-00000
2014-04-10 11:46:34,968 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@20d020d
2014-04-10 11:46:34,969 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:34,969 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:34,979 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:34,979 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:34,979 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:34,979 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:34,979 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:34,980 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:34,980 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:34,983 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][4315789487/source/]
2014-04-10 11:46:34,983 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2014-04-10 11:46:34,984 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:34,984 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:34,984 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:34,984 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 65; bufvoid = 104857600
2014-04-10 11:46:34,984 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2014-04-10 11:46:34,985 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:34,986 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1285614220_0032_m_000001_0 is done. And is in the process of committing
2014-04-10 11:46:34,987 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-brock/4315789487_source_27ABE25713A0429A9EB8DCA7E343CE75/part-00000:0+160
2014-04-10 11:46:34,987 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1285614220_0032_m_000001_0' done.
2014-04-10 11:46:34,987 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1285614220_0032_m_000001_0
2014-04-10 11:46:34,987 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:34,987 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:34,987 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1285614220_0032_r_000000_0
2014-04-10 11:46:34,989 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:34,989 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@45096a91
2014-04-10 11:46:34,990 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:34,990 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1285614220_0032_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:34,991 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#25 about to shuffle output of map attempt_local1285614220_0032_m_000001_0 decomp: 77 len: 81 to MEMORY
2014-04-10 11:46:34,992 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 77 bytes from map-output for attempt_local1285614220_0032_m_000001_0
2014-04-10 11:46:34,992 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 77, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->77
2014-04-10 11:46:34,994 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#25 about to shuffle output of map attempt_local1285614220_0032_m_000000_0 decomp: 77 len: 81 to MEMORY
2014-04-10 11:46:34,994 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 77 bytes from map-output for attempt_local1285614220_0032_m_000000_0
2014-04-10 11:46:34,994 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 77, inMemoryMapOutputs.size() -> 2, commitMemory -> 77, usedMemory ->154
2014-04-10 11:46:34,995 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:34,996 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:34,996 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:34,997 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2014-04-10 11:46:34,998 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 134 bytes
2014-04-10 11:46:34,999 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 154 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:34,999 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 156 bytes from disk
2014-04-10 11:46:34,999 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:35,000 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:35,000 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 142 bytes
2014-04-10 11:46:35,000 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:35,001 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:35,001 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:35,009 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2014-04-10 11:46:35,009 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: TempHfs["SequenceFile[['char', 'num', 'char2', 'count']]"][6610859707/source_count-rhs/]
2014-04-10 11:46:35,016 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(105)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2014-04-10 11:46:35,016 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(110)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2014-04-10 11:46:35,021 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1285614220_0032_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:35,021 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:35,021 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1285614220_0032_r_000000_0 is allowed to commit now
2014-04-10 11:46:35,022 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1285614220_0032_r_000000_0' to file:/tmp/hadoop-brock/6610859707_source_count_rhs_501C12B68FC14B269F707F9982FE120C/_temporary/0/task_local1285614220_0032_r_000000
2014-04-10 11:46:35,022 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:35,022 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1285614220_0032_r_000000_0' done.
2014-04-10 11:46:35,023 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1285614220_0032_r_000000_0
2014-04-10 11:46:35,023 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:35,037 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (5/5) ...m/samesourcemergethreeway
2014-04-10 11:46:35,037 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:35,037 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:35,054 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:35,060 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:35,069 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2014-04-10 11:46:35,075 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:35,075 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:35,080 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local855696585_0033
2014-04-10 11:46:35,100 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock855696585/.staging/job_local855696585_0033/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:35,100 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock855696585/.staging/job_local855696585_0033/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:35,124 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local855696585_0033/job_local855696585_0033.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:35,124 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local855696585_0033/job_local855696585_0033.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:35,125 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:35,125 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local855696585_0033
2014-04-10 11:46:35,125 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:35,125 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2014-04-10 11:46:35,125 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:35,127 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:35,128 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local855696585_0033_m_000000_0
2014-04-10 11:46:35,129 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:35,130 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-brock/6610859707_source_count_rhs_501C12B68FC14B269F707F9982FE120C/part-00000
2014-04-10 11:46:35,130 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5a3c0970
2014-04-10 11:46:35,131 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:35,131 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:35,144 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:35,144 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:35,144 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:35,144 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:35,144 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:35,145 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:35,145 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:35,148 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: TempHfs["SequenceFile[['char', 'num', 'char2', 'count']]"][6610859707/source_count-rhs/]
2014-04-10 11:46:35,148 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2014-04-10 11:46:35,149 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:35,149 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:35,149 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:35,149 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 105; bufvoid = 104857600
2014-04-10 11:46:35,149 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2014-04-10 11:46:35,150 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:35,151 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local855696585_0033_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:35,151 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-brock/6610859707_source_count_rhs_501C12B68FC14B269F707F9982FE120C/part-00000:0+200
2014-04-10 11:46:35,152 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local855696585_0033_m_000000_0' done.
2014-04-10 11:46:35,152 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local855696585_0033_m_000000_0
2014-04-10 11:46:35,152 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local855696585_0033_m_000001_0
2014-04-10 11:46:35,152 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:35,153 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-brock/3285150057_count_lhs_CE5F24697C984271850830A75D0B8D69/part-00000
2014-04-10 11:46:35,153 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2a2a5dc3
2014-04-10 11:46:35,153 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:35,153 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:35,164 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:35,164 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:35,164 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:35,164 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:35,164 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:35,165 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:35,165 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:35,168 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][3285150057/count-lhs/]
2014-04-10 11:46:35,168 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2014-04-10 11:46:35,169 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:35,169 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:35,169 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:35,169 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 65; bufvoid = 104857600
2014-04-10 11:46:35,169 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2014-04-10 11:46:35,170 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:35,171 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local855696585_0033_m_000001_0 is done. And is in the process of committing
2014-04-10 11:46:35,171 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-brock/3285150057_count_lhs_CE5F24697C984271850830A75D0B8D69/part-00000:0+160
2014-04-10 11:46:35,171 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local855696585_0033_m_000001_0' done.
2014-04-10 11:46:35,172 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local855696585_0033_m_000001_0
2014-04-10 11:46:35,172 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:35,172 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:35,172 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local855696585_0033_r_000000_0
2014-04-10 11:46:35,173 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:35,173 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@2736ef65
2014-04-10 11:46:35,173 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:35,173 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local855696585_0033_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:35,175 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#26 about to shuffle output of map attempt_local855696585_0033_m_000000_0 decomp: 117 len: 121 to MEMORY
2014-04-10 11:46:35,176 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 117 bytes from map-output for attempt_local855696585_0033_m_000000_0
2014-04-10 11:46:35,176 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 117, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->117
2014-04-10 11:46:35,177 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#26 about to shuffle output of map attempt_local855696585_0033_m_000001_0 decomp: 77 len: 81 to MEMORY
2014-04-10 11:46:35,178 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 77 bytes from map-output for attempt_local855696585_0033_m_000001_0
2014-04-10 11:46:35,178 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 77, inMemoryMapOutputs.size() -> 2, commitMemory -> 117, usedMemory ->194
2014-04-10 11:46:35,179 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:35,179 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:35,180 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:35,181 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2014-04-10 11:46:35,181 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 174 bytes
2014-04-10 11:46:35,182 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 194 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:35,182 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 196 bytes from disk
2014-04-10 11:46:35,182 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:35,183 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:35,183 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 182 bytes
2014-04-10 11:46:35,183 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:35,184 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:35,184 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:35,189 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2014-04-10 11:46:35,189 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway"]
2014-04-10 11:46:35,193 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(105)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2014-04-10 11:46:35,194 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(110)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2014-04-10 11:46:35,196 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local855696585_0033_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:35,197 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:35,198 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local855696585_0033_r_000000_0 is allowed to commit now
2014-04-10 11:46:35,198 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local855696585_0033_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway/_temporary/0/task_local855696585_0033_r_000000
2014-04-10 11:46:35,199 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:35,199 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local855696585_0033_r_000000_0' done.
2014-04-10 11:46:35,199 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local855696585_0033_r_000000_0
2014-04-10 11:46:35,199 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:35,210 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway/_temporary
2014-04-10 11:46:35,235 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:35,255 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:35,257 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway
2014-04-10 11:46:35,266 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:35,266 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:35,270 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:35,271 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:35,271 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:35,275 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [lhs+rhs] starting
2014-04-10 11:46:35,275 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [lhs+rhs]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt"]
2014-04-10 11:46:35,276 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [lhs+rhs]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:35,276 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [lhs+rhs]  sink: Hfs["TextLine[['item']->['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic"]
2014-04-10 11:46:35,276 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [lhs+rhs]  parallel execution is enabled: false
2014-04-10 11:46:35,276 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [lhs+rhs]  starting jobs: 1
2014-04-10 11:46:35,276 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [lhs+rhs]  allocating threads: 1
2014-04-10 11:46:35,277 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lhs+rhs] starting step: (1/1) ...quemerge-nondeterministic
2014-04-10 11:46:35,277 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:35,278 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:35,290 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:35,291 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:35,301 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2014-04-10 11:46:35,312 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local340556701_0034
2014-04-10 11:46:35,323 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock340556701/.staging/job_local340556701_0034/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:35,323 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock340556701/.staging/job_local340556701_0034/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:35,351 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local340556701_0034/job_local340556701_0034.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:35,351 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local340556701_0034/job_local340556701_0034.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:35,351 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:35,351 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lhs+rhs] submitted hadoop job: job_local340556701_0034
2014-04-10 11:46:35,352 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:35,352 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lhs+rhs] tracking url: http://localhost:8080/
2014-04-10 11:46:35,352 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:35,354 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:35,354 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local340556701_0034_m_000000_0
2014-04-10 11:46:35,356 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:35,357 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:35,357 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6c2745b7
2014-04-10 11:46:35,358 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:35,358 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:35,369 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:35,369 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:35,369 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:35,369 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:35,370 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:35,370 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:35,370 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:35,373 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:35,373 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2014-04-10 11:46:35,375 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:35,375 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:35,375 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:35,375 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2014-04-10 11:46:35,375 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2014-04-10 11:46:35,376 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:35,376 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local340556701_0034_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:35,377 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:35,377 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local340556701_0034_m_000000_0' done.
2014-04-10 11:46:35,377 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local340556701_0034_m_000000_0
2014-04-10 11:46:35,377 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local340556701_0034_m_000001_0
2014-04-10 11:46:35,378 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:35,378 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt
2014-04-10 11:46:35,378 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6e67f937
2014-04-10 11:46:35,379 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:35,379 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:35,391 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:35,391 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:35,391 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:35,391 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:35,391 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:35,392 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:35,392 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:35,394 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt"]
2014-04-10 11:46:35,394 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2014-04-10 11:46:35,396 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:35,396 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:35,396 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:35,396 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2014-04-10 11:46:35,396 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2014-04-10 11:46:35,397 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:35,397 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local340556701_0034_m_000001_0 is done. And is in the process of committing
2014-04-10 11:46:35,398 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/rhs.txt:0+51
2014-04-10 11:46:35,398 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local340556701_0034_m_000001_0' done.
2014-04-10 11:46:35,398 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local340556701_0034_m_000001_0
2014-04-10 11:46:35,398 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:35,399 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:35,399 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local340556701_0034_r_000000_0
2014-04-10 11:46:35,399 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:35,399 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@713a902c
2014-04-10 11:46:35,400 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:35,400 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local340556701_0034_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:35,401 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#27 about to shuffle output of map attempt_local340556701_0034_m_000001_0 decomp: 87 len: 91 to MEMORY
2014-04-10 11:46:35,401 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local340556701_0034_m_000001_0
2014-04-10 11:46:35,401 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2014-04-10 11:46:35,403 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#27 about to shuffle output of map attempt_local340556701_0034_m_000000_0 decomp: 87 len: 91 to MEMORY
2014-04-10 11:46:35,403 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local340556701_0034_m_000000_0
2014-04-10 11:46:35,404 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 2, commitMemory -> 87, usedMemory ->174
2014-04-10 11:46:35,404 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:35,405 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:35,405 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:35,406 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2014-04-10 11:46:35,406 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 156 bytes
2014-04-10 11:46:35,406 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 174 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:35,406 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 176 bytes from disk
2014-04-10 11:46:35,406 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:35,406 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:35,407 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 163 bytes
2014-04-10 11:46:35,407 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:35,407 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:35,407 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:35,410 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2014-04-10 11:46:35,410 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextLine[['item']->['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic"]
2014-04-10 11:46:35,416 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local340556701_0034_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:35,418 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 11:46:35,418 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local340556701_0034_r_000000_0 is allowed to commit now
2014-04-10 11:46:35,419 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local340556701_0034_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic/_temporary/0/task_local340556701_0034_r_000000
2014-04-10 11:46:35,419 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:35,419 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local340556701_0034_r_000000_0' done.
2014-04-10 11:46:35,419 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local340556701_0034_r_000000_0
2014-04-10 11:46:35,420 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:35,438 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic/_temporary
2014-04-10 11:46:35,458 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:35,459 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic
2014-04-10 11:46:35,472 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:35,472 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:35,474 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:35,475 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:35,475 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:35,478 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] starting
2014-04-10 11:46:35,478 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:35,478 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  sink: Hfs["TextLine[['item']->['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique"]
2014-04-10 11:46:35,478 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  parallel execution is enabled: false
2014-04-10 11:46:35,478 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  starting jobs: 1
2014-04-10 11:46:35,478 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [shape]  allocating threads: 1
2014-04-10 11:46:35,479 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...blyhelpersplatform/unique
2014-04-10 11:46:35,480 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:35,481 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:35,492 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:35,501 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:35,511 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1525480103_0035
2014-04-10 11:46:35,522 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1525480103/.staging/job_local1525480103_0035/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:35,522 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1525480103/.staging/job_local1525480103_0035/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:35,540 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1525480103_0035/job_local1525480103_0035.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:35,541 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1525480103_0035/job_local1525480103_0035.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:35,541 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:35,541 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1525480103_0035
2014-04-10 11:46:35,541 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:35,542 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2014-04-10 11:46:35,542 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:35,544 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:35,544 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1525480103_0035_m_000000_0
2014-04-10 11:46:35,546 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:35,547 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:35,547 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5902f207
2014-04-10 11:46:35,548 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:35,548 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:35,566 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:35,566 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:35,566 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:35,566 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:35,566 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:35,568 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:35,568 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:35,570 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:35,570 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(shape)[by:[{1}:'num']]
2014-04-10 11:46:35,571 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:35,571 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:35,571 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:35,571 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2014-04-10 11:46:35,571 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2014-04-10 11:46:35,572 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:35,573 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1525480103_0035_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:35,574 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:35,574 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1525480103_0035_m_000000_0' done.
2014-04-10 11:46:35,574 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1525480103_0035_m_000000_0
2014-04-10 11:46:35,574 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:35,575 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:35,575 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1525480103_0035_r_000000_0
2014-04-10 11:46:35,576 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:35,576 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7c0e843
2014-04-10 11:46:35,577 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:35,577 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1525480103_0035_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:35,579 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#28 about to shuffle output of map attempt_local1525480103_0035_m_000000_0 decomp: 87 len: 91 to MEMORY
2014-04-10 11:46:35,579 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local1525480103_0035_m_000000_0
2014-04-10 11:46:35,580 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2014-04-10 11:46:35,581 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:35,582 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:35,582 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:35,584 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:35,584 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2014-04-10 11:46:35,585 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 87 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:35,585 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 91 bytes from disk
2014-04-10 11:46:35,585 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:35,586 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:35,586 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2014-04-10 11:46:35,587 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:35,587 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:35,587 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:35,590 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(shape)[by:[{1}:'num']]
2014-04-10 11:46:35,590 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextLine[['item']->['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique"]
2014-04-10 11:46:35,593 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1525480103_0035_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:35,593 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:35,593 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1525480103_0035_r_000000_0 is allowed to commit now
2014-04-10 11:46:35,594 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1525480103_0035_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique/_temporary/0/task_local1525480103_0035_r_000000
2014-04-10 11:46:35,594 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:35,594 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1525480103_0035_r_000000_0' done.
2014-04-10 11:46:35,594 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1525480103_0035_r_000000_0
2014-04-10 11:46:35,594 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:35,607 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique/_temporary
2014-04-10 11:46:35,621 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:35,622 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/unique
2014-04-10 11:46:35,639 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 11:46:35,639 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 11:46:35,642 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 11:46:35,643 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 11:46:35,643 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 11:46:35,646 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count] starting
2014-04-10 11:46:35,646 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count]  source: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:35,646 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count]  sink: Hfs["TextDelimited[['notnull', 'null']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull"]
2014-04-10 11:46:35,646 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count]  parallel execution is enabled: false
2014-04-10 11:46:35,647 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count]  starting jobs: 1
2014-04-10 11:46:35,647 INFO  flow.Flow (BaseFlow.java:logInfo(1354)) - [count]  allocating threads: 1
2014-04-10 11:46:35,647 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...platform/countnullnotnull
2014-04-10 11:46:35,648 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:35,649 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 11:46:35,659 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:35,665 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 11:46:35,675 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1978480505_0036
2014-04-10 11:46:35,689 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1978480505/.staging/job_local1978480505_0036/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:35,689 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1978480505/.staging/job_local1978480505_0036/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:35,708 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1978480505_0036/job_local1978480505_0036.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-04-10 11:46:35,708 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1978480505_0036/job_local1978480505_0036.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-04-10 11:46:35,709 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 11:46:35,709 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local1978480505_0036
2014-04-10 11:46:35,709 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2014-04-10 11:46:35,709 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 11:46:35,709 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 11:46:35,711 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 11:46:35,711 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1978480505_0036_m_000000_0
2014-04-10 11:46:35,712 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:35,713 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt
2014-04-10 11:46:35,713 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@51f6292b
2014-04-10 11:46:35,714 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 11:46:35,714 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 11:46:35,731 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 11:46:35,731 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 11:46:35,731 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 11:46:35,731 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 11:46:35,731 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 11:46:35,732 INFO  hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 11:46:35,732 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 11:46:35,735 INFO  hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt"]
2014-04-10 11:46:35,735 INFO  hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(count)[by:[{?}:NONE]]
2014-04-10 11:46:35,735 INFO  assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-10 11:46:35,738 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2014-04-10 11:46:35,738 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 11:46:35,738 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 11:46:35,738 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 6; bufvoid = 104857600
2014-04-10 11:46:35,738 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214396(104857584); length = 1/6553600
2014-04-10 11:46:35,739 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 11:46:35,740 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1978480505_0036_m_000000_0 is done. And is in the process of committing
2014-04-10 11:46:35,740 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lhs.txt:0+52
2014-04-10 11:46:35,740 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1978480505_0036_m_000000_0' done.
2014-04-10 11:46:35,740 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1978480505_0036_m_000000_0
2014-04-10 11:46:35,741 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 11:46:35,741 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 11:46:35,741 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1978480505_0036_r_000000_0
2014-04-10 11:46:35,743 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2014-04-10 11:46:35,743 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@6aaf6af2
2014-04-10 11:46:35,743 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 11:46:35,744 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1978480505_0036_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 11:46:35,745 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#29 about to shuffle output of map attempt_local1978480505_0036_m_000000_0 decomp: 10 len: 14 to MEMORY
2014-04-10 11:46:35,745 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 10 bytes from map-output for attempt_local1978480505_0036_m_000000_0
2014-04-10 11:46:35,745 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 10, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->10
2014-04-10 11:46:35,746 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 11:46:35,747 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:35,747 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 11:46:35,748 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:35,748 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 7 bytes
2014-04-10 11:46:35,749 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 10 bytes to disk to satisfy reduce memory limit
2014-04-10 11:46:35,750 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 14 bytes from disk
2014-04-10 11:46:35,750 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 11:46:35,750 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 11:46:35,750 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 7 bytes
2014-04-10 11:46:35,751 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:35,751 INFO  hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 11:46:35,751 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 11:46:35,758 INFO  hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(count)[by:[{?}:NONE]]
2014-04-10 11:46:35,758 INFO  hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['notnull', 'null']]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull"]
2014-04-10 11:46:35,762 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1978480505_0036_r_000000_0 is done. And is in the process of committing
2014-04-10 11:46:35,763 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 11:46:35,764 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1978480505_0036_r_000000_0 is allowed to commit now
2014-04-10 11:46:35,764 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1978480505_0036_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull/_temporary/0/task_local1978480505_0036_r_000000
2014-04-10 11:46:35,765 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 11:46:35,765 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1978480505_0036_r_000000_0' done.
2014-04-10 11:46:35,765 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1978480505_0036_r_000000_0
2014-04-10 11:46:35,766 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 11:46:35,784 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull/_temporary
2014-04-10 11:46:35,805 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:35,825 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 11:46:35,826 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/assemblyhelpersplatform/countnullnotnull