Class cascading.pipe.assembly.AssemblyHelpersPlatformTest

all > cascading.pipe.assembly > AssemblyHelpersPlatformTest
tests
failures
ignored
22.265s
duration
100%
successful
Tests
Standard output
Tests

Test	Duration	Result
testAverageBy	0.406s	passed
testAverageByNull	0.325s	passed
testAverageMerge	1.788s	passed
testCoerce	0.227s	passed
testCoerceFields	0.612s	passed
testCount	0.546s	passed
testCountAll	0.302s	passed
testCountCount	1.918s	passed
testCountMerge	1.082s	passed
testCountNullNotNull	0.308s	passed
testDiscardNarrow	0.296s	passed
testFirstBy	0.638s	passed
testFirstByWithoutComparator	0.260s	passed
testMaxBy	0.745s	passed
testMaxByNullSafety	0.577s	passed
testMaxByString	0.589s	passed
testMinBy	0.655s	passed
testMinByNullSafety	0.630s	passed
testMinByString	1.297s	passed
testParallelAggregates	0.235s	passed
testParallelAggregatesMerge	3.045s	passed
testParallelAggregatesPriorMerge	0.561s	passed
testRenameAll	0.488s	passed
testRenameNamed	0.306s	passed
testRenameNarrow	0.182s	passed
testRetainNarrow	0.156s	passed
testSameSourceMerge	0.747s	passed
testSameSourceMergeThreeWay	1.251s	passed
testSumBy	0.629s	passed
testSumByNulls	0.441s	passed
testSumMerge	0.465s	passed
testUnique	0.259s	passed
testUniqueMerge	0.299s	passed
Standard output

2015-02-12 12:03:51,371 INFO  platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(187)) - found 1 test platforms from classpath
2015-02-12 12:03:51,415 INFO  platform.PlatformRunner (PlatformRunner.java:addPlatform(237)) - installing platform: hadoop2-mr1
2015-02-12 12:03:51,415 INFO  platform.PlatformRunner (PlatformRunner.java:addPlatform(238)) - running test: cascading.pipe.assembly.AssemblyHelpersPlatformTest
2015-02-12 12:03:51,445 INFO  hadoop2.Hadoop2MR1Platform (Hadoop2MR1Platform.java:setUp(105)) - not using cluster
2015-02-12 12:03:52,194 WARN  util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-02-12 12:03:53,035 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:03:53,037 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:03:53,065 INFO  property.AppProps (AppProps.java:getAppID(169)) - using app.id: 3CFF6C1825E74A0C83970F65B1DB746B
2015-02-12 12:03:53,276 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:03:53,309 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:03:53,312 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:03:53,409 INFO  util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 2.6.3
2015-02-12 12:03:53,410 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name] starting
2015-02-12 12:03:53,411 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt"]
2015-02-12 12:03:53,411 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:03:53,411 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false"]
2015-02-12 12:03:53,412 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  parallel execution is enabled: false
2015-02-12 12:03:53,412 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  starting jobs: 1
2015-02-12 12:03:53,412 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  allocating threads: 1
2015-02-12 12:03:53,413 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] starting step: (1/1) ...platform/multimerge+false
2015-02-12 12:03:53,425 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - session.id is deprecated. Instead, use dfs.metrics.session-id
2015-02-12 12:03:53,425 INFO  jvm.JvmMetrics (JvmMetrics.java:init(76)) - Initializing JVM Metrics with processName=JobTracker, sessionId=
2015-02-12 12:03:53,437 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:03:53,580 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:03:53,586 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:03:53,606 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:2
2015-02-12 12:03:53,747 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local82696083_0001
2015-02-12 12:03:53,888 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:03:53,889 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:03:53,890 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] submitted hadoop job: job_local82696083_0001
2015-02-12 12:03:53,890 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:03:53,890 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] tracking url: http://localhost:8080/
2015-02-12 12:03:53,919 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:03:53,921 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local82696083_0001_m_000000_0
2015-02-12 12:03:53,941 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:03:53,942 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:03:53,947 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:03:53,947 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@65e7e8bf
2015-02-12 12:03:53,953 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:03:54,007 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:03:54,007 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:03:54,007 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:03:54,007 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:03:54,008 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:03:54,028 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:03:54,029 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:03:54,030 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:03:54,063 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
2015-02-12 12:03:54,068 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:03:54,068 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2015-02-12 12:03:54,070 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:03:54,079 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:03:54,079 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 86, total: 245, max: 672
2015-02-12 12:03:54,079 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 36% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:03:54,080 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:03:54,080 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 86, total: 245, max: 672
2015-02-12 12:03:54,080 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 36% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:03:54,081 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:03:54,081 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 86, total: 245, max: 672
2015-02-12 12:03:54,081 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 36% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:03:54,082 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:03:54,082 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 86, total: 245, max: 672
2015-02-12 12:03:54,082 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 36% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:03:54,085 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:03:54,085 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:03:54,085 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:03:54,086 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2015-02-12 12:03:54,086 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:03:54,091 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:03:54,094 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local82696083_0001_m_000000_0 is done. And is in the process of committing
2015-02-12 12:03:54,101 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:03:54,101 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local82696083_0001_m_000000_0' done.
2015-02-12 12:03:54,101 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local82696083_0001_m_000000_0
2015-02-12 12:03:54,102 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local82696083_0001_m_000001_0
2015-02-12 12:03:54,105 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:03:54,105 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:03:54,107 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt
2015-02-12 12:03:54,107 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@fef7a08
2015-02-12 12:03:54,109 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:03:54,164 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:03:54,164 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:03:54,164 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:03:54,165 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:03:54,165 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:03:54,166 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:03:54,167 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:03:54,167 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:03:54,186 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt"]
2015-02-12 12:03:54,186 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2015-02-12 12:03:54,187 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:03:54,253 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:03:54,254 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 81, total: 346, max: 672
2015-02-12 12:03:54,254 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 51% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:03:54,255 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:03:54,255 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 81, total: 346, max: 672
2015-02-12 12:03:54,255 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 51% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:03:54,256 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:03:54,256 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 81, total: 346, max: 672
2015-02-12 12:03:54,256 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 51% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:03:54,257 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:03:54,257 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 81, total: 346, max: 672
2015-02-12 12:03:54,257 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 51% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:03:54,258 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:03:54,259 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:03:54,259 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:03:54,259 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2015-02-12 12:03:54,259 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:03:54,261 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:03:54,263 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local82696083_0001_m_000001_0 is done. And is in the process of committing
2015-02-12 12:03:54,265 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt:0+51
2015-02-12 12:03:54,266 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local82696083_0001_m_000001_0' done.
2015-02-12 12:03:54,266 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local82696083_0001_m_000001_0
2015-02-12 12:03:54,266 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:03:54,268 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:03:54,269 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local82696083_0001_r_000000_0
2015-02-12 12:03:54,275 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:03:54,275 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:03:54,278 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5baeceb
2015-02-12 12:03:54,287 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:03:54,289 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local82696083_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:03:54,306 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#1 about to shuffle output of map attempt_local82696083_0001_m_000000_0 decomp: 236 len: 240 to MEMORY
2015-02-12 12:03:54,309 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local82696083_0001_m_000000_0
2015-02-12 12:03:54,342 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->236
2015-02-12 12:03:54,344 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#1 about to shuffle output of map attempt_local82696083_0001_m_000001_0 decomp: 236 len: 240 to MEMORY
2015-02-12 12:03:54,344 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local82696083_0001_m_000001_0
2015-02-12 12:03:54,345 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 2, commitMemory -> 236, usedMemory ->472
2015-02-12 12:03:54,345 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:03:54,346 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:03:54,346 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:03:54,351 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 2 sorted segments
2015-02-12 12:03:54,352 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 2 segments left of total size: 454 bytes
2015-02-12 12:03:54,353 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 2 segments, 472 bytes to disk to satisfy reduce memory limit
2015-02-12 12:03:54,353 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 474 bytes from disk
2015-02-12 12:03:54,354 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:03:54,354 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:03:54,355 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 461 bytes
2015-02-12 12:03:54,355 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:03:54,357 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:03:54,357 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:03:54,373 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2015-02-12 12:03:54,373 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false"]
2015-02-12 12:03:54,389 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local82696083_0001_r_000000_0 is done. And is in the process of committing
2015-02-12 12:03:54,391 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:03:54,391 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local82696083_0001_r_000000_0 is allowed to commit now
2015-02-12 12:03:54,392 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local82696083_0001_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false/_temporary/0/task_local82696083_0001_r_000000
2015-02-12 12:03:54,393 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:03:54,393 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local82696083_0001_r_000000_0' done.
2015-02-12 12:03:54,394 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local82696083_0001_r_000000_0
2015-02-12 12:03:54,394 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:03:54,428 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false/_temporary
2015-02-12 12:03:54,456 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:03:54,486 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:03:54,488 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/multimerge+false
2015-02-12 12:03:54,509 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:03:54,510 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:03:54,518 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:03:54,521 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:03:54,522 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:03:54,531 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting
2015-02-12 12:03:54,531 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:03:54,531 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  sink: Hfs["TextDelimited[['num', 'min']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring"]
2015-02-12 12:03:54,531 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  parallel execution is enabled: false
2015-02-12 12:03:54,531 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  starting jobs: 1
2015-02-12 12:03:54,532 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  allocating threads: 1
2015-02-12 12:03:54,532 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...lpersplatform/minbystring
2015-02-12 12:03:54,532 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:03:54,534 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:03:54,557 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:03:54,578 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:03:54,599 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local1061971144_0002
2015-02-12 12:03:54,679 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:03:54,679 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:03:54,679 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:03:54,679 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local1061971144_0002
2015-02-12 12:03:54,679 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2015-02-12 12:03:54,682 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:03:54,682 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1061971144_0002_m_000000_0
2015-02-12 12:03:54,684 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:03:54,684 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:03:54,686 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:03:54,686 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@a4861f2
2015-02-12 12:03:54,689 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:03:54,852 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:03:54,852 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:03:54,853 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:03:54,853 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:03:54,853 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:03:54,864 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:03:54,872 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:03:54,872 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:03:54,899 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:03:54,900 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'num']]
2015-02-12 12:03:54,901 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:03:54,905 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:03:54,905 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 79, total: 446, max: 672
2015-02-12 12:03:54,906 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 66% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:03:54,907 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:03:54,907 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 79, total: 446, max: 672
2015-02-12 12:03:54,907 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 66% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:03:55,013 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:03:55,013 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:03:55,014 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:03:55,014 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-12 12:03:55,014 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-12 12:03:55,016 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:03:55,018 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1061971144_0002_m_000000_0 is done. And is in the process of committing
2015-02-12 12:03:55,020 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:03:55,021 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1061971144_0002_m_000000_0' done.
2015-02-12 12:03:55,021 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1061971144_0002_m_000000_0
2015-02-12 12:03:55,021 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:03:55,022 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:03:55,022 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1061971144_0002_r_000000_0
2015-02-12 12:03:55,024 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:03:55,025 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:03:55,025 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5d708401
2015-02-12 12:03:55,026 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:03:55,026 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1061971144_0002_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:03:55,028 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#2 about to shuffle output of map attempt_local1061971144_0002_m_000000_0 decomp: 87 len: 91 to MEMORY
2015-02-12 12:03:55,028 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local1061971144_0002_m_000000_0
2015-02-12 12:03:55,029 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2015-02-12 12:03:55,030 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:03:55,030 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:03:55,031 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:03:55,032 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:03:55,032 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-12 12:03:55,034 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 87 bytes to disk to satisfy reduce memory limit
2015-02-12 12:03:55,034 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 91 bytes from disk
2015-02-12 12:03:55,034 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:03:55,034 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:03:55,035 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-12 12:03:55,035 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:03:55,036 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:03:55,036 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:03:55,076 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'num']]
2015-02-12 12:03:55,076 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'min']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring"]
2015-02-12 12:03:55,113 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1061971144_0002_r_000000_0 is done. And is in the process of committing
2015-02-12 12:03:55,120 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:03:55,121 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local1061971144_0002_r_000000_0 is allowed to commit now
2015-02-12 12:03:55,122 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1061971144_0002_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring/_temporary/0/task_local1061971144_0002_r_000000
2015-02-12 12:03:55,128 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:03:55,129 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1061971144_0002_r_000000_0' done.
2015-02-12 12:03:55,129 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1061971144_0002_r_000000_0
2015-02-12 12:03:55,131 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:03:55,170 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring/_temporary
2015-02-12 12:03:55,487 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:03:55,783 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:03:55,784 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/minbystring
2015-02-12 12:03:56,085 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:03:56,086 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:03:56,166 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:03:56,256 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:03:56,257 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:03:56,303 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs] starting
2015-02-12 12:03:56,303 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt"]
2015-02-12 12:03:56,304 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:03:56,304 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs]  sink: Hfs["TextDelimited[['char', 'average']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage"]
2015-02-12 12:03:56,304 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs]  parallel execution is enabled: false
2015-02-12 12:03:56,304 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs]  starting jobs: 1
2015-02-12 12:03:56,304 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs]  allocating threads: 1
2015-02-12 12:03:56,305 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average-lhs+average-rhs] starting step: (1/1) ...persplatform/mergeaverage
2015-02-12 12:03:56,305 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:03:56,307 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:03:56,345 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:03:56,348 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:03:56,380 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:2
2015-02-12 12:03:56,588 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local1571621645_0003
2015-02-12 12:03:57,041 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:03:57,042 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:03:57,042 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:03:57,043 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average-lhs+average-rhs] submitted hadoop job: job_local1571621645_0003
2015-02-12 12:03:57,043 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average-lhs+average-rhs] tracking url: http://localhost:8080/
2015-02-12 12:03:57,045 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:03:57,046 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1571621645_0003_m_000000_0
2015-02-12 12:03:57,047 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:03:57,047 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:03:57,049 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:03:57,049 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@15c673a9
2015-02-12 12:03:57,050 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:03:57,112 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:03:57,112 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:03:57,112 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:03:57,112 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:03:57,113 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:03:57,114 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:03:57,115 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:03:57,115 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:03:57,133 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:03:57,133 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2015-02-12 12:03:57,134 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:03:57,135 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:03:57,135 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 151, total: 611, max: 672
2015-02-12 12:03:57,136 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:03:57,136 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 151, total: 611, max: 672
2015-02-12 12:03:57,137 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:03:57,137 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 151, total: 611, max: 672
2015-02-12 12:03:57,138 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:03:57,138 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 151, total: 611, max: 672
2015-02-12 12:03:57,139 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:03:57,139 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:03:57,140 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:03:57,140 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2015-02-12 12:03:57,140 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:03:57,142 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:03:57,144 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1571621645_0003_m_000000_0 is done. And is in the process of committing
2015-02-12 12:03:57,146 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:03:57,147 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1571621645_0003_m_000000_0' done.
2015-02-12 12:03:57,147 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1571621645_0003_m_000000_0
2015-02-12 12:03:57,147 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1571621645_0003_m_000001_0
2015-02-12 12:03:57,149 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:03:57,149 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:03:57,150 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt
2015-02-12 12:03:57,151 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@213b91a5
2015-02-12 12:03:57,152 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:03:57,303 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:03:57,303 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:03:57,304 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:03:57,306 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:03:57,306 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:03:57,316 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:03:57,327 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:03:57,327 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:03:57,341 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt"]
2015-02-12 12:03:57,341 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2015-02-12 12:03:57,347 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:03:57,350 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:03:57,350 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 81, total: 643, max: 672
2015-02-12 12:03:57,351 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:03:57,351 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 81, total: 643, max: 672
2015-02-12 12:03:57,352 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:03:57,352 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 81, total: 643, max: 672
2015-02-12 12:03:57,353 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:03:57,353 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 81, total: 643, max: 672
2015-02-12 12:03:57,354 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:03:57,354 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:03:57,354 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:03:57,355 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2015-02-12 12:03:57,378 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:03:57,381 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:03:57,383 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1571621645_0003_m_000001_0 is done. And is in the process of committing
2015-02-12 12:03:57,385 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt:0+51
2015-02-12 12:03:57,386 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1571621645_0003_m_000001_0' done.
2015-02-12 12:03:57,386 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1571621645_0003_m_000001_0
2015-02-12 12:03:57,386 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:03:57,387 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:03:57,387 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1571621645_0003_r_000000_0
2015-02-12 12:03:57,389 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:03:57,389 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:03:57,389 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@16516ac3
2015-02-12 12:03:57,391 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:03:57,392 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1571621645_0003_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:03:57,394 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#3 about to shuffle output of map attempt_local1571621645_0003_m_000000_0 decomp: 200 len: 204 to MEMORY
2015-02-12 12:03:57,394 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local1571621645_0003_m_000000_0
2015-02-12 12:03:57,394 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->200
2015-02-12 12:03:57,396 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#3 about to shuffle output of map attempt_local1571621645_0003_m_000001_0 decomp: 200 len: 204 to MEMORY
2015-02-12 12:03:57,397 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local1571621645_0003_m_000001_0
2015-02-12 12:03:57,397 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 2, commitMemory -> 200, usedMemory ->400
2015-02-12 12:03:57,397 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:03:57,398 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:03:57,398 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:03:57,400 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 2 sorted segments
2015-02-12 12:03:57,400 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 2 segments left of total size: 382 bytes
2015-02-12 12:03:57,401 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 2 segments, 400 bytes to disk to satisfy reduce memory limit
2015-02-12 12:03:57,402 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 402 bytes from disk
2015-02-12 12:03:57,402 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:03:57,402 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:03:57,402 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 389 bytes
2015-02-12 12:03:57,403 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:03:57,403 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:03:57,403 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:03:57,413 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2015-02-12 12:03:57,413 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage"]
2015-02-12 12:03:57,425 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1571621645_0003_r_000000_0 is done. And is in the process of committing
2015-02-12 12:03:57,427 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:03:57,427 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local1571621645_0003_r_000000_0 is allowed to commit now
2015-02-12 12:03:57,428 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1571621645_0003_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage/_temporary/0/task_local1571621645_0003_r_000000
2015-02-12 12:03:57,429 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:03:57,429 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1571621645_0003_r_000000_0' done.
2015-02-12 12:03:57,429 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1571621645_0003_r_000000_0
2015-02-12 12:03:57,429 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:03:57,472 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage/_temporary
2015-02-12 12:03:57,525 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:03:57,572 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:03:57,575 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/mergeaverage
2015-02-12 12:03:57,606 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:03:57,607 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:03:57,614 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:03:57,617 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:03:57,618 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:03:57,627 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-12 12:03:57,628 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt"]
2015-02-12 12:03:57,628 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  sink: Hfs["TextLine[['line']->['item', 'element']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall"]
2015-02-12 12:03:57,628 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  parallel execution is enabled: false
2015-02-12 12:03:57,628 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  starting jobs: 1
2015-02-12 12:03:57,629 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  allocating threads: 1
2015-02-12 12:03:57,629 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...helpersplatform/renameall
2015-02-12 12:03:57,630 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:03:57,631 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:03:57,670 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:03:57,697 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:03:57,721 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local1596027994_0004
2015-02-12 12:03:57,947 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:03:57,948 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:03:57,948 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:03:57,948 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1596027994_0004
2015-02-12 12:03:57,948 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-12 12:03:57,950 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:03:57,950 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1596027994_0004_m_000000_0
2015-02-12 12:03:57,953 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:03:57,953 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:03:57,954 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt
2015-02-12 12:03:57,955 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@45bbd91
2015-02-12 12:03:57,956 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 0
2015-02-12 12:03:57,965 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:03:57,965 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:03:57,971 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt"]
2015-02-12 12:03:57,972 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['item', 'element']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall"]
2015-02-12 12:03:57,975 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:03:57,976 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1596027994_0004_m_000000_0 is done. And is in the process of committing
2015-02-12 12:03:57,977 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:03:57,977 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local1596027994_0004_m_000000_0 is allowed to commit now
2015-02-12 12:03:57,978 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1596027994_0004_m_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall/_temporary/0/task_local1596027994_0004_m_000000
2015-02-12 12:03:57,979 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt:0+19
2015-02-12 12:03:57,979 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1596027994_0004_m_000000_0' done.
2015-02-12 12:03:57,980 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1596027994_0004_m_000000_0
2015-02-12 12:03:57,980 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:03:58,017 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall/_temporary
2015-02-12 12:03:58,063 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:03:58,065 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/renameall
2015-02-12 12:03:58,107 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:03:58,109 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:03:58,442 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:03:58,446 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:03:58,464 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:03:58,481 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting
2015-02-12 12:03:58,481 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:03:58,481 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  sink: Hfs["TextDelimited[['count', 'count2']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount"]
2015-02-12 12:03:58,506 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  parallel execution is enabled: false
2015-02-12 12:03:58,507 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  starting jobs: 2
2015-02-12 12:03:58,507 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  allocating threads: 1
2015-02-12 12:03:58,507 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/2)
2015-02-12 12:03:58,508 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:03:58,510 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:03:58,546 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:03:58,573 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:03:58,593 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local80208945_0005
2015-02-12 12:03:59,041 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:03:59,042 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:03:59,042 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:03:59,042 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local80208945_0005
2015-02-12 12:03:59,043 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-12 12:03:59,045 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:03:59,045 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local80208945_0005_m_000000_0
2015-02-12 12:03:59,046 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:03:59,047 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:03:59,048 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:03:59,048 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@18b5a36e
2015-02-12 12:03:59,049 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:03:59,062 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:03:59,063 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:03:59,063 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:03:59,063 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:03:59,063 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:03:59,064 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:03:59,064 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:03:59,064 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:03:59,073 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:03:59,074 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{1}:'char']]
2015-02-12 12:03:59,074 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:03:59,075 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:03:59,075 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 124, total: 269, max: 672
2015-02-12 12:03:59,076 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 40% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:03:59,076 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:03:59,076 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 124, total: 269, max: 672
2015-02-12 12:03:59,076 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 40% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:03:59,077 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:03:59,077 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 124, total: 269, max: 672
2015-02-12 12:03:59,077 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 40% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:03:59,078 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:03:59,078 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 124, total: 269, max: 672
2015-02-12 12:03:59,078 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 40% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:03:59,079 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:03:59,079 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:03:59,079 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:03:59,080 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-12 12:03:59,080 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:03:59,082 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:03:59,084 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local80208945_0005_m_000000_0 is done. And is in the process of committing
2015-02-12 12:03:59,086 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:03:59,086 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local80208945_0005_m_000000_0' done.
2015-02-12 12:03:59,086 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local80208945_0005_m_000000_0
2015-02-12 12:03:59,086 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:03:59,087 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:03:59,087 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local80208945_0005_r_000000_0
2015-02-12 12:03:59,089 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:03:59,089 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:03:59,089 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@41c7d592
2015-02-12 12:03:59,090 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:03:59,090 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local80208945_0005_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:03:59,092 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#4 about to shuffle output of map attempt_local80208945_0005_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-12 12:03:59,093 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local80208945_0005_m_000000_0
2015-02-12 12:03:59,093 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-12 12:03:59,093 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:03:59,094 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:03:59,094 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:03:59,096 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:03:59,096 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-12 12:03:59,097 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-12 12:03:59,097 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 123 bytes from disk
2015-02-12 12:03:59,098 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:03:59,098 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:03:59,098 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-12 12:03:59,099 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:03:59,099 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:03:59,099 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:03:59,108 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{1}:'char']]
2015-02-12 12:03:59,108 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['count', 'count2']]"][4388928916/count/]
2015-02-12 12:03:59,129 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:03:59,132 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:03:59,132 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 119, total: 269, max: 672
2015-02-12 12:03:59,132 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 40% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:03:59,133 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local80208945_0005_r_000000_0 is done. And is in the process of committing
2015-02-12 12:03:59,134 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:03:59,134 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local80208945_0005_r_000000_0 is allowed to commit now
2015-02-12 12:03:59,135 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local80208945_0005_r_000000_0' to file:/tmp/hadoop-soberoi/4388928916_count_FC9806CF042E44A2AB5C5E8A989234A5/_temporary/0/task_local80208945_0005_r_000000
2015-02-12 12:03:59,137 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:03:59,137 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local80208945_0005_r_000000_0' done.
2015-02-12 12:03:59,137 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local80208945_0005_r_000000_0
2015-02-12 12:03:59,137 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:03:59,167 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (2/2) ...elpersplatform/countcount
2015-02-12 12:03:59,168 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:03:59,169 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:03:59,225 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:03:59,256 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:03:59,274 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:03:59,275 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:03:59,368 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local1973842545_0006
2015-02-12 12:03:59,752 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:03:59,753 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:03:59,753 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:03:59,753 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local1973842545_0006
2015-02-12 12:03:59,753 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-12 12:03:59,755 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:03:59,755 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1973842545_0006_m_000000_0
2015-02-12 12:03:59,756 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:03:59,757 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:03:59,758 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-soberoi/4388928916_count_FC9806CF042E44A2AB5C5E8A989234A5/part-00000
2015-02-12 12:03:59,758 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@47287f45
2015-02-12 12:03:59,767 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:03:59,817 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:03:59,818 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:03:59,818 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:03:59,818 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:03:59,818 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:03:59,819 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:03:59,819 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:03:59,819 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:03:59,828 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['count', 'count2']]"][4388928916/count/]
2015-02-12 12:03:59,829 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{1}:'count']]
2015-02-12 12:03:59,831 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:03:59,831 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:03:59,831 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:03:59,831 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 21; bufvoid = 104857600
2015-02-12 12:03:59,831 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214388(104857552); length = 9/6553600
2015-02-12 12:03:59,833 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:03:59,835 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1973842545_0006_m_000000_0 is done. And is in the process of committing
2015-02-12 12:03:59,837 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-soberoi/4388928916_count_FC9806CF042E44A2AB5C5E8A989234A5/part-00000:0+112
2015-02-12 12:03:59,837 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1973842545_0006_m_000000_0' done.
2015-02-12 12:03:59,837 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1973842545_0006_m_000000_0
2015-02-12 12:03:59,837 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:03:59,838 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:03:59,838 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1973842545_0006_r_000000_0
2015-02-12 12:03:59,840 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:03:59,840 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:03:59,840 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@3d57369e
2015-02-12 12:03:59,840 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:03:59,841 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1973842545_0006_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:03:59,842 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#5 about to shuffle output of map attempt_local1973842545_0006_m_000000_0 decomp: 29 len: 33 to MEMORY
2015-02-12 12:03:59,843 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 29 bytes from map-output for attempt_local1973842545_0006_m_000000_0
2015-02-12 12:03:59,843 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 29, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->29
2015-02-12 12:03:59,843 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:03:59,844 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:03:59,844 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:03:59,846 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:03:59,846 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 24 bytes
2015-02-12 12:03:59,847 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 29 bytes to disk to satisfy reduce memory limit
2015-02-12 12:03:59,847 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 33 bytes from disk
2015-02-12 12:03:59,847 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:03:59,847 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:03:59,848 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 24 bytes
2015-02-12 12:03:59,848 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:03:59,849 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:03:59,849 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:03:59,857 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{1}:'count']]
2015-02-12 12:03:59,858 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['count', 'count2']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount"]
2015-02-12 12:03:59,869 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1973842545_0006_r_000000_0 is done. And is in the process of committing
2015-02-12 12:03:59,872 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:03:59,872 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local1973842545_0006_r_000000_0 is allowed to commit now
2015-02-12 12:03:59,873 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1973842545_0006_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount/_temporary/0/task_local1973842545_0006_r_000000
2015-02-12 12:03:59,874 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:03:59,874 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1973842545_0006_r_000000_0' done.
2015-02-12 12:03:59,874 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1973842545_0006_r_000000_0
2015-02-12 12:03:59,874 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:03:59,899 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount/_temporary
2015-02-12 12:03:59,940 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:03:59,982 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:03:59,984 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/countcount
2015-02-12 12:04:00,082 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:00,113 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:00,158 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:00,178 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:00,179 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:00,256 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs] starting
2015-02-12 12:04:00,257 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt"]
2015-02-12 12:04:00,257 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:00,257 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs]  sink: Hfs["TextDelimited[['char', 'count']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount"]
2015-02-12 12:04:00,257 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs]  parallel execution is enabled: false
2015-02-12 12:04:00,257 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs]  starting jobs: 1
2015-02-12 12:04:00,257 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs]  allocating threads: 1
2015-02-12 12:04:00,258 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs+count-rhs] starting step: (1/1) ...elpersplatform/mergecount
2015-02-12 12:04:00,258 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:00,259 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:00,291 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:00,292 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:00,319 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:2
2015-02-12 12:04:00,339 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local269753674_0007
2015-02-12 12:04:00,616 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:00,616 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:00,616 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:00,618 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs+count-rhs] submitted hadoop job: job_local269753674_0007
2015-02-12 12:04:00,618 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs+count-rhs] tracking url: http://localhost:8080/
2015-02-12 12:04:00,619 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local269753674_0007_m_000000_0
2015-02-12 12:04:00,619 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:00,621 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:00,622 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:00,623 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:04:00,623 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5ab16c52
2015-02-12 12:04:00,624 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:00,638 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:00,638 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:00,638 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:00,638 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:00,639 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:00,639 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:00,639 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:00,640 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:00,649 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:00,649 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-12 12:04:00,650 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:00,651 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:00,651 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 48, total: 370, max: 672
2015-02-12 12:04:00,652 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 55% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:04:00,652 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:00,652 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 48, total: 370, max: 672
2015-02-12 12:04:00,652 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 55% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:04:00,653 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:04:00,653 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 48, total: 370, max: 672
2015-02-12 12:04:00,653 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 55% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:04:00,653 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:04:00,654 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 48, total: 370, max: 672
2015-02-12 12:04:00,654 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 55% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:04:00,655 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:00,655 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:00,655 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:00,655 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-12 12:04:00,655 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:04:00,657 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:00,659 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local269753674_0007_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:00,661 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:04:00,661 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local269753674_0007_m_000000_0' done.
2015-02-12 12:04:00,661 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local269753674_0007_m_000000_0
2015-02-12 12:04:00,661 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local269753674_0007_m_000001_0
2015-02-12 12:04:00,663 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:00,663 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:00,665 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt
2015-02-12 12:04:00,665 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@19930f2a
2015-02-12 12:04:00,666 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:00,822 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:00,822 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:00,822 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:00,822 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:00,822 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:00,823 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:00,824 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:00,824 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:00,846 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt"]
2015-02-12 12:04:00,858 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-12 12:04:00,886 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:00,889 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:00,890 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 47, total: 470, max: 672
2015-02-12 12:04:00,890 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:04:00,891 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:00,891 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 47, total: 470, max: 672
2015-02-12 12:04:00,891 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:04:00,892 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:04:00,892 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 47, total: 470, max: 672
2015-02-12 12:04:00,892 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:04:00,893 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:04:00,893 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 47, total: 470, max: 672
2015-02-12 12:04:00,893 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 69% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:04:00,894 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:00,894 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:00,894 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:00,894 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-12 12:04:00,894 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:04:00,897 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:00,899 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local269753674_0007_m_000001_0 is done. And is in the process of committing
2015-02-12 12:04:00,901 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt:0+51
2015-02-12 12:04:00,901 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local269753674_0007_m_000001_0' done.
2015-02-12 12:04:00,901 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local269753674_0007_m_000001_0
2015-02-12 12:04:00,901 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:00,902 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:00,902 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local269753674_0007_r_000000_0
2015-02-12 12:04:00,904 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:00,904 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:00,904 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5a52ae21
2015-02-12 12:04:00,905 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:00,905 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local269753674_0007_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:00,907 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#6 about to shuffle output of map attempt_local269753674_0007_m_000001_0 decomp: 119 len: 123 to MEMORY
2015-02-12 12:04:00,908 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local269753674_0007_m_000001_0
2015-02-12 12:04:00,908 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-12 12:04:00,909 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#6 about to shuffle output of map attempt_local269753674_0007_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-12 12:04:00,910 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local269753674_0007_m_000000_0
2015-02-12 12:04:00,910 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 2, commitMemory -> 119, usedMemory ->238
2015-02-12 12:04:00,910 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:00,911 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:04:00,911 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:00,913 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 2 sorted segments
2015-02-12 12:04:00,913 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 2 segments left of total size: 220 bytes
2015-02-12 12:04:00,915 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 2 segments, 238 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:00,915 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 240 bytes from disk
2015-02-12 12:04:00,915 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:00,915 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:00,916 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 227 bytes
2015-02-12 12:04:00,916 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:04:00,917 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:00,917 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:00,927 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-12 12:04:00,927 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount"]
2015-02-12 12:04:00,938 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local269753674_0007_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:00,939 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:04:00,940 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local269753674_0007_r_000000_0 is allowed to commit now
2015-02-12 12:04:00,940 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local269753674_0007_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount/_temporary/0/task_local269753674_0007_r_000000
2015-02-12 12:04:00,942 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:00,942 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local269753674_0007_r_000000_0' done.
2015-02-12 12:04:00,942 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local269753674_0007_r_000000_0
2015-02-12 12:04:00,942 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:00,970 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount/_temporary
2015-02-12 12:04:01,006 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:01,065 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:01,066 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/mergecount
2015-02-12 12:04:01,130 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:01,130 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:01,139 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:01,155 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:01,182 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:01,201 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min] starting
2015-02-12 12:04:01,201 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:01,201 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  sink: Hfs["TextDelimited[['char', 'min']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbynullsafety"]
2015-02-12 12:04:01,201 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  parallel execution is enabled: false
2015-02-12 12:04:01,201 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  starting jobs: 1
2015-02-12 12:04:01,201 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  allocating threads: 1
2015-02-12 12:04:01,202 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] starting step: (1/1) ...splatform/minbynullsafety
2015-02-12 12:04:01,202 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:01,203 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:01,234 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:01,268 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:01,291 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local218980514_0008
2015-02-12 12:04:01,469 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:01,469 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:01,470 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:01,470 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] submitted hadoop job: job_local218980514_0008
2015-02-12 12:04:01,471 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] tracking url: http://localhost:8080/
2015-02-12 12:04:01,472 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:01,472 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local218980514_0008_m_000000_0
2015-02-12 12:04:01,473 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:01,473 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:01,475 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:04:01,475 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@285eaab5
2015-02-12 12:04:01,476 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:01,493 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:01,493 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:01,494 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:01,494 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:01,494 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:01,494 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:01,495 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:01,495 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:01,503 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:01,503 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(min)[by:[{1}:'char']]
2015-02-12 12:04:01,504 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:01,505 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:01,505 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 54, total: 512, max: 672
2015-02-12 12:04:01,506 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 76% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:04:01,506 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:01,506 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 54, total: 512, max: 672
2015-02-12 12:04:01,507 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 76% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:04:01,507 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:04:01,507 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 54, total: 512, max: 672
2015-02-12 12:04:01,507 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 76% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:04:01,508 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:04:01,508 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 54, total: 512, max: 672
2015-02-12 12:04:01,508 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 76% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:04:01,509 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:01,509 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:01,509 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:01,509 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-12 12:04:01,509 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:04:01,514 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:01,515 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local218980514_0008_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:01,517 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:04:01,517 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local218980514_0008_m_000000_0' done.
2015-02-12 12:04:01,517 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local218980514_0008_m_000000_0
2015-02-12 12:04:01,517 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:01,518 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:01,518 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local218980514_0008_r_000000_0
2015-02-12 12:04:01,519 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:01,520 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:01,520 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@25187150
2015-02-12 12:04:01,520 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:01,521 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local218980514_0008_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:01,522 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#7 about to shuffle output of map attempt_local218980514_0008_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-12 12:04:01,523 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local218980514_0008_m_000000_0
2015-02-12 12:04:01,523 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-12 12:04:01,524 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:01,524 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:01,525 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:01,526 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:01,526 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-12 12:04:01,527 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:01,527 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 123 bytes from disk
2015-02-12 12:04:01,528 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:01,528 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:01,528 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-12 12:04:01,529 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:01,529 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:01,529 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:01,538 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(min)[by:[{1}:'char']]
2015-02-12 12:04:01,538 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'min']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbynullsafety"]
2015-02-12 12:04:01,551 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local218980514_0008_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:01,553 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:01,553 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local218980514_0008_r_000000_0 is allowed to commit now
2015-02-12 12:04:01,554 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local218980514_0008_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbynullsafety/_temporary/0/task_local218980514_0008_r_000000
2015-02-12 12:04:01,555 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:01,556 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local218980514_0008_r_000000_0' done.
2015-02-12 12:04:01,556 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local218980514_0008_r_000000_0
2015-02-12 12:04:01,556 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:01,584 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbynullsafety/_temporary
2015-02-12 12:04:01,622 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:01,695 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:01,697 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/minbynullsafety
2015-02-12 12:04:01,734 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:01,767 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:01,776 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:01,778 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:01,778 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:01,787 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting
2015-02-12 12:04:01,788 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:01,788 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  sink: Hfs["TextDelimited[['num', 'max']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring"]
2015-02-12 12:04:01,788 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  parallel execution is enabled: false
2015-02-12 12:04:01,788 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  starting jobs: 1
2015-02-12 12:04:01,788 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  allocating threads: 1
2015-02-12 12:04:01,788 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...lpersplatform/maxbystring
2015-02-12 12:04:01,789 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:01,790 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:01,820 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:01,845 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:01,865 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local295366802_0009
2015-02-12 12:04:02,040 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:02,040 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:02,040 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local295366802_0009
2015-02-12 12:04:02,040 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:02,040 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2015-02-12 12:04:02,042 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:02,042 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local295366802_0009_m_000000_0
2015-02-12 12:04:02,044 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:02,044 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:02,045 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:04:02,045 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@111648d0
2015-02-12 12:04:02,046 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:02,059 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:02,060 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:02,060 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:02,060 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:02,060 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:02,061 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:02,061 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:02,061 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:02,068 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:02,069 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'num']]
2015-02-12 12:04:02,069 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:02,070 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:02,071 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 59, total: 514, max: 672
2015-02-12 12:04:02,071 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 76% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:04:02,071 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:02,071 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 59, total: 514, max: 672
2015-02-12 12:04:02,072 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 76% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 12:04:02,073 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:02,073 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:02,073 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:02,073 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-12 12:04:02,073 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-12 12:04:02,075 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:02,076 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local295366802_0009_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:02,078 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:04:02,078 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local295366802_0009_m_000000_0' done.
2015-02-12 12:04:02,078 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local295366802_0009_m_000000_0
2015-02-12 12:04:02,079 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:02,079 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:02,079 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local295366802_0009_r_000000_0
2015-02-12 12:04:02,081 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:02,081 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:02,081 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@172ea70d
2015-02-12 12:04:02,082 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:02,082 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local295366802_0009_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:02,084 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#8 about to shuffle output of map attempt_local295366802_0009_m_000000_0 decomp: 87 len: 91 to MEMORY
2015-02-12 12:04:02,084 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local295366802_0009_m_000000_0
2015-02-12 12:04:02,084 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2015-02-12 12:04:02,085 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:02,085 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:02,086 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:02,087 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:02,087 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-12 12:04:02,088 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 87 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:02,088 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 91 bytes from disk
2015-02-12 12:04:02,088 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:02,088 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:02,089 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-12 12:04:02,089 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:02,089 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:02,089 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:02,096 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'num']]
2015-02-12 12:04:02,097 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'max']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring"]
2015-02-12 12:04:02,108 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local295366802_0009_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:02,109 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:02,109 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local295366802_0009_r_000000_0 is allowed to commit now
2015-02-12 12:04:02,110 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local295366802_0009_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring/_temporary/0/task_local295366802_0009_r_000000
2015-02-12 12:04:02,111 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:02,112 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local295366802_0009_r_000000_0' done.
2015-02-12 12:04:02,112 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local295366802_0009_r_000000_0
2015-02-12 12:04:02,112 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:02,140 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring/_temporary
2015-02-12 12:04:02,180 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:02,284 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:02,286 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/maxbystring
2015-02-12 12:04:02,399 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:02,400 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:02,427 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:02,429 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:02,429 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:02,445 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first] starting
2015-02-12 12:04:02,445 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  source: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 12:04:02,445 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  sink: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfields"]
2015-02-12 12:04:02,445 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  parallel execution is enabled: false
2015-02-12 12:04:02,445 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  starting jobs: 1
2015-02-12 12:04:02,445 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  allocating threads: 1
2015-02-12 12:04:02,446 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] starting step: (1/1) ...persplatform/firstnfields
2015-02-12 12:04:02,446 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:02,449 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:02,478 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:02,505 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:02,527 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local642053543_0010
2015-02-12 12:04:02,768 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:02,769 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:02,769 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:02,771 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] submitted hadoop job: job_local642053543_0010
2015-02-12 12:04:02,771 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] tracking url: http://localhost:8080/
2015-02-12 12:04:02,771 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:02,771 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local642053543_0010_m_000000_0
2015-02-12 12:04:02,773 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:02,773 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:02,774 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 12:04:02,774 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@59bd80fc
2015-02-12 12:04:02,775 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:02,789 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:02,789 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:02,789 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:02,789 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:02,789 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:02,791 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:02,792 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:02,792 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:02,799 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 12:04:02,799 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(first)[by:[{1}:'num']]
2015-02-12 12:04:02,800 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:02,803 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:02,803 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 112, total: 569, max: 672
2015-02-12 12:04:02,804 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:02,804 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 112, total: 569, max: 672
2015-02-12 12:04:02,806 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:02,806 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:02,806 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:02,806 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 170; bufvoid = 104857600
2015-02-12 12:04:02,806 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-12 12:04:02,808 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:02,809 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local642053543_0010_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:02,811 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 12:04:02,811 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local642053543_0010_m_000000_0' done.
2015-02-12 12:04:02,811 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local642053543_0010_m_000000_0
2015-02-12 12:04:02,811 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:02,812 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:02,812 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local642053543_0010_r_000000_0
2015-02-12 12:04:02,814 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:02,814 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:02,814 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@10417813
2015-02-12 12:04:02,814 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:02,815 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local642053543_0010_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:02,816 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#9 about to shuffle output of map attempt_local642053543_0010_m_000000_0 decomp: 182 len: 186 to MEMORY
2015-02-12 12:04:02,817 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 182 bytes from map-output for attempt_local642053543_0010_m_000000_0
2015-02-12 12:04:02,817 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 182, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->182
2015-02-12 12:04:02,828 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:02,829 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:02,829 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:02,830 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:02,830 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 160 bytes
2015-02-12 12:04:02,831 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 182 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:02,832 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 186 bytes from disk
2015-02-12 12:04:02,832 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:02,832 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:02,832 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 160 bytes
2015-02-12 12:04:02,833 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:02,833 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:02,834 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:02,840 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(first)[by:[{1}:'num']]
2015-02-12 12:04:02,840 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfields"]
2015-02-12 12:04:02,855 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local642053543_0010_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:02,856 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:02,857 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local642053543_0010_r_000000_0 is allowed to commit now
2015-02-12 12:04:02,857 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local642053543_0010_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfields/_temporary/0/task_local642053543_0010_r_000000
2015-02-12 12:04:02,859 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:02,859 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local642053543_0010_r_000000_0' done.
2015-02-12 12:04:02,859 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local642053543_0010_r_000000_0
2015-02-12 12:04:02,859 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:02,878 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfields/_temporary
2015-02-12 12:04:02,921 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:02,923 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/firstnfields
2015-02-12 12:04:02,999 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:03,000 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:03,025 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:03,032 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:03,033 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:03,046 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] starting
2015-02-12 12:04:03,046 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt"]
2015-02-12 12:04:03,047 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  sink: Hfs["TextLine[['line']->['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields"]
2015-02-12 12:04:03,047 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  parallel execution is enabled: false
2015-02-12 12:04:03,047 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  starting jobs: 1
2015-02-12 12:04:03,047 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  allocating threads: 1
2015-02-12 12:04:03,047 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] starting step: (1/1) ...persplatform/coercefields
2015-02-12 12:04:03,048 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:03,049 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:03,080 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:03,106 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:03,128 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local1712360727_0011
2015-02-12 12:04:03,426 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:03,427 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:03,427 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:03,427 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] submitted hadoop job: job_local1712360727_0011
2015-02-12 12:04:03,427 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] tracking url: http://localhost:8080/
2015-02-12 12:04:03,429 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:03,429 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1712360727_0011_m_000000_0
2015-02-12 12:04:03,431 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:03,431 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:03,432 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt
2015-02-12 12:04:03,432 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@509ce6c4
2015-02-12 12:04:03,433 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 0
2015-02-12 12:04:03,440 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:03,440 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:03,448 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt"]
2015-02-12 12:04:03,449 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields"]
2015-02-12 12:04:03,457 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:03,458 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1712360727_0011_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:03,459 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:03,459 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local1712360727_0011_m_000000_0 is allowed to commit now
2015-02-12 12:04:03,460 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1712360727_0011_m_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields/_temporary/0/task_local1712360727_0011_m_000000
2015-02-12 12:04:03,461 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt:0+19
2015-02-12 12:04:03,461 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1712360727_0011_m_000000_0' done.
2015-02-12 12:04:03,461 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1712360727_0011_m_000000_0
2015-02-12 12:04:03,462 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:03,486 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields/_temporary
2015-02-12 12:04:03,534 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:03,536 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/coercefields
2015-02-12 12:04:03,577 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:03,578 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:03,583 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:03,585 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:03,586 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:03,592 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting
2015-02-12 12:04:03,593 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:03,593 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  sink: Hfs["TextDelimited[['char', 'count']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count"]
2015-02-12 12:04:03,593 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  parallel execution is enabled: false
2015-02-12 12:04:03,593 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  starting jobs: 1
2015-02-12 12:04:03,593 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  allocating threads: 1
2015-02-12 12:04:03,593 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...mblyhelpersplatform/count
2015-02-12 12:04:03,594 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:03,595 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:03,625 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:03,653 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:03,676 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local834959766_0012
2015-02-12 12:04:03,907 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:03,908 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:03,908 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:03,908 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local834959766_0012
2015-02-12 12:04:03,908 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-12 12:04:03,910 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:03,910 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local834959766_0012_m_000000_0
2015-02-12 12:04:03,911 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:03,911 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:03,912 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:04:03,913 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@34659b02
2015-02-12 12:04:03,914 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:03,927 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:03,927 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:03,927 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:03,927 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:03,927 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:03,928 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:03,928 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:03,928 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:03,935 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:03,936 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{1}:'char']]
2015-02-12 12:04:03,936 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:03,937 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:03,937 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 101, total: 559, max: 672
2015-02-12 12:04:03,938 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:03,938 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 101, total: 559, max: 672
2015-02-12 12:04:03,938 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:04:03,939 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 101, total: 559, max: 672
2015-02-12 12:04:03,939 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:04:03,939 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 101, total: 559, max: 672
2015-02-12 12:04:03,940 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:03,940 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:03,940 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:03,940 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-12 12:04:03,940 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:04:03,942 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:03,944 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local834959766_0012_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:03,946 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:04:03,946 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local834959766_0012_m_000000_0' done.
2015-02-12 12:04:03,946 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local834959766_0012_m_000000_0
2015-02-12 12:04:03,946 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:03,946 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:03,947 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local834959766_0012_r_000000_0
2015-02-12 12:04:03,948 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:03,948 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:03,948 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@1b9ee7a5
2015-02-12 12:04:03,949 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:03,950 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local834959766_0012_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:03,951 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#10 about to shuffle output of map attempt_local834959766_0012_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-12 12:04:03,951 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local834959766_0012_m_000000_0
2015-02-12 12:04:03,952 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-12 12:04:03,952 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:03,953 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:03,953 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:03,955 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:03,955 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-12 12:04:03,956 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:03,956 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 123 bytes from disk
2015-02-12 12:04:03,956 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:03,956 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:03,957 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-12 12:04:03,957 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:03,957 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:03,957 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:03,964 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{1}:'char']]
2015-02-12 12:04:03,965 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count"]
2015-02-12 12:04:03,978 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local834959766_0012_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:03,980 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:03,980 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local834959766_0012_r_000000_0 is allowed to commit now
2015-02-12 12:04:03,981 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local834959766_0012_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count/_temporary/0/task_local834959766_0012_r_000000
2015-02-12 12:04:03,982 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:03,982 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local834959766_0012_r_000000_0' done.
2015-02-12 12:04:03,982 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local834959766_0012_r_000000_0
2015-02-12 12:04:03,982 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:04,005 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count/_temporary
2015-02-12 12:04:04,042 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:04,081 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:04,082 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/count
2015-02-12 12:04:04,205 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:04,205 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:04,213 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:04,215 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:04,216 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:04,301 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting
2015-02-12 12:04:04,301 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:04,301 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  sink: Hfs["TextDelimited[['char', 'max']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby"]
2015-02-12 12:04:04,301 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  parallel execution is enabled: false
2015-02-12 12:04:04,301 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  starting jobs: 1
2015-02-12 12:04:04,302 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  allocating threads: 1
2015-02-12 12:04:04,302 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...mblyhelpersplatform/maxby
2015-02-12 12:04:04,303 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:04,303 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:04,333 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:04,361 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:04,380 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local1919068744_0013
2015-02-12 12:04:04,649 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:04,649 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:04,650 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:04,650 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local1919068744_0013
2015-02-12 12:04:04,650 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2015-02-12 12:04:04,652 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:04,652 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1919068744_0013_m_000000_0
2015-02-12 12:04:04,653 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:04,653 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:04,655 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:04:04,655 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2f6da05c
2015-02-12 12:04:04,656 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:04,673 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:04,673 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:04,673 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:04,673 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:04,673 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:04,676 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:04,677 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:04,677 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:04,684 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:04,685 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'char']]
2015-02-12 12:04:04,685 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:04,686 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:04,686 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 155, total: 569, max: 672
2015-02-12 12:04:04,687 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:04,687 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 155, total: 569, max: 672
2015-02-12 12:04:04,688 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:04:04,688 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 155, total: 569, max: 672
2015-02-12 12:04:04,688 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:04:04,688 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 155, total: 569, max: 672
2015-02-12 12:04:04,690 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:04,690 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:04,690 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:04,690 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 135; bufvoid = 104857600
2015-02-12 12:04:04,690 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:04:04,692 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:04,693 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1919068744_0013_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:04,695 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:04:04,695 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1919068744_0013_m_000000_0' done.
2015-02-12 12:04:04,695 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1919068744_0013_m_000000_0
2015-02-12 12:04:04,696 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:04,696 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:04,696 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1919068744_0013_r_000000_0
2015-02-12 12:04:04,698 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:04,698 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:04,698 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@28e66cff
2015-02-12 12:04:04,698 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:04,699 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1919068744_0013_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:04,700 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#11 about to shuffle output of map attempt_local1919068744_0013_m_000000_0 decomp: 155 len: 159 to MEMORY
2015-02-12 12:04:04,701 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 155 bytes from map-output for attempt_local1919068744_0013_m_000000_0
2015-02-12 12:04:04,701 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 155, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->155
2015-02-12 12:04:04,701 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:04,702 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:04,702 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:04,704 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:04,704 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2015-02-12 12:04:04,705 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 155 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:04,705 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 159 bytes from disk
2015-02-12 12:04:04,705 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:04,705 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:04,705 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2015-02-12 12:04:04,706 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:04,706 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:04,706 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:04,713 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'char']]
2015-02-12 12:04:04,713 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'max']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby"]
2015-02-12 12:04:04,724 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1919068744_0013_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:04,726 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:04,726 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local1919068744_0013_r_000000_0 is allowed to commit now
2015-02-12 12:04:04,727 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1919068744_0013_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby/_temporary/0/task_local1919068744_0013_r_000000
2015-02-12 12:04:04,728 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:04,728 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1919068744_0013_r_000000_0' done.
2015-02-12 12:04:04,728 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1919068744_0013_r_000000_0
2015-02-12 12:04:04,728 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:04,755 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby/_temporary
2015-02-12 12:04:04,792 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:04,826 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:04,828 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/maxby
2015-02-12 12:04:04,923 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:04,924 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:04,932 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:04,934 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:04,934 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:04,945 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min] starting
2015-02-12 12:04:04,945 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:04,945 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  sink: Hfs["TextDelimited[['char', 'min']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby"]
2015-02-12 12:04:04,945 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  parallel execution is enabled: false
2015-02-12 12:04:04,945 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  starting jobs: 1
2015-02-12 12:04:04,946 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  allocating threads: 1
2015-02-12 12:04:04,946 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] starting step: (1/1) ...mblyhelpersplatform/minby
2015-02-12 12:04:04,947 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:04,948 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:04,977 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:05,002 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:05,022 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local1131740744_0014
2015-02-12 12:04:05,254 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:05,254 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:05,254 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:05,254 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] submitted hadoop job: job_local1131740744_0014
2015-02-12 12:04:05,254 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] tracking url: http://localhost:8080/
2015-02-12 12:04:05,257 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:05,257 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1131740744_0014_m_000000_0
2015-02-12 12:04:05,258 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:05,258 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:05,259 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:04:05,260 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7b5e8a9e
2015-02-12 12:04:05,261 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:05,288 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:05,288 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:05,288 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:05,288 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:05,288 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:05,289 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:05,289 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:05,289 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:05,296 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:05,296 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(min)[by:[{1}:'char']]
2015-02-12 12:04:05,297 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:05,298 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:05,298 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 156, total: 569, max: 672
2015-02-12 12:04:05,299 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:05,299 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 156, total: 569, max: 672
2015-02-12 12:04:05,299 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:04:05,300 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 156, total: 569, max: 672
2015-02-12 12:04:05,300 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:04:05,300 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 156, total: 569, max: 672
2015-02-12 12:04:05,301 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:05,301 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:05,301 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:05,302 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 135; bufvoid = 104857600
2015-02-12 12:04:05,302 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:04:05,304 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:05,305 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1131740744_0014_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:05,307 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:04:05,307 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1131740744_0014_m_000000_0' done.
2015-02-12 12:04:05,307 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1131740744_0014_m_000000_0
2015-02-12 12:04:05,307 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:05,308 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:05,308 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1131740744_0014_r_000000_0
2015-02-12 12:04:05,309 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:05,310 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:05,310 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7abfe56d
2015-02-12 12:04:05,310 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:05,311 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1131740744_0014_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:05,312 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#12 about to shuffle output of map attempt_local1131740744_0014_m_000000_0 decomp: 155 len: 159 to MEMORY
2015-02-12 12:04:05,312 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 155 bytes from map-output for attempt_local1131740744_0014_m_000000_0
2015-02-12 12:04:05,312 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 155, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->155
2015-02-12 12:04:05,313 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:05,314 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:05,314 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:05,315 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:05,315 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2015-02-12 12:04:05,316 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 155 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:05,317 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 159 bytes from disk
2015-02-12 12:04:05,317 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:05,317 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:05,317 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2015-02-12 12:04:05,318 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:05,318 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:05,318 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:05,324 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(min)[by:[{1}:'char']]
2015-02-12 12:04:05,324 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'min']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby"]
2015-02-12 12:04:05,335 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1131740744_0014_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:05,336 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:05,336 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local1131740744_0014_r_000000_0 is allowed to commit now
2015-02-12 12:04:05,337 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1131740744_0014_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby/_temporary/0/task_local1131740744_0014_r_000000
2015-02-12 12:04:05,338 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:05,338 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1131740744_0014_r_000000_0' done.
2015-02-12 12:04:05,339 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1131740744_0014_r_000000_0
2015-02-12 12:04:05,339 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:05,356 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby/_temporary
2015-02-12 12:04:05,400 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:05,481 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:05,483 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/minby
2015-02-12 12:04:05,543 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:05,543 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:05,592 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:05,593 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:05,594 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:05,654 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] starting
2015-02-12 12:04:05,655 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:05,655 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  sink: Hfs["TextDelimited[['char', 'sum']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum"]
2015-02-12 12:04:05,655 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  parallel execution is enabled: false
2015-02-12 12:04:05,655 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  starting jobs: 1
2015-02-12 12:04:05,655 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  allocating threads: 1
2015-02-12 12:04:05,655 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] starting step: (1/1) ...semblyhelpersplatform/sum
2015-02-12 12:04:05,656 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:05,657 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:05,688 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:05,714 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:05,733 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local2077901671_0015
2015-02-12 12:04:05,922 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:05,923 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:05,923 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:05,923 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] submitted hadoop job: job_local2077901671_0015
2015-02-12 12:04:05,923 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] tracking url: http://localhost:8080/
2015-02-12 12:04:05,925 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:05,925 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local2077901671_0015_m_000000_0
2015-02-12 12:04:05,927 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:05,927 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:05,928 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:04:05,928 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3e8bb77b
2015-02-12 12:04:05,929 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:05,957 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:05,957 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:05,957 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:05,957 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:05,958 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:05,958 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:05,959 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:05,959 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:05,966 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:05,966 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum)[by:[{1}:'char']]
2015-02-12 12:04:05,966 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:05,968 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:05,968 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 156, total: 569, max: 672
2015-02-12 12:04:05,969 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:05,969 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 156, total: 569, max: 672
2015-02-12 12:04:05,969 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:04:05,969 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 156, total: 569, max: 672
2015-02-12 12:04:05,970 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:04:05,970 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 156, total: 569, max: 672
2015-02-12 12:04:05,971 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:05,971 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:05,971 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:05,971 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-12 12:04:05,972 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:04:05,974 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:05,975 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local2077901671_0015_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:05,977 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:04:05,977 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local2077901671_0015_m_000000_0' done.
2015-02-12 12:04:05,977 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local2077901671_0015_m_000000_0
2015-02-12 12:04:05,977 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:05,977 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:05,978 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local2077901671_0015_r_000000_0
2015-02-12 12:04:05,979 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:05,979 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:05,979 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5e854128
2015-02-12 12:04:05,980 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:05,980 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local2077901671_0015_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:05,981 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#13 about to shuffle output of map attempt_local2077901671_0015_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-12 12:04:05,982 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local2077901671_0015_m_000000_0
2015-02-12 12:04:05,982 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-12 12:04:05,982 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:05,983 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:05,983 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:05,984 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:05,985 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-12 12:04:05,985 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:05,986 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 123 bytes from disk
2015-02-12 12:04:05,986 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:05,986 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:05,986 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-12 12:04:05,987 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:05,987 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:05,987 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:05,994 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(sum)[by:[{1}:'char']]
2015-02-12 12:04:05,994 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum"]
2015-02-12 12:04:06,005 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local2077901671_0015_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:06,006 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:06,007 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local2077901671_0015_r_000000_0 is allowed to commit now
2015-02-12 12:04:06,007 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local2077901671_0015_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum/_temporary/0/task_local2077901671_0015_r_000000
2015-02-12 12:04:06,008 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:06,009 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local2077901671_0015_r_000000_0' done.
2015-02-12 12:04:06,009 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local2077901671_0015_r_000000_0
2015-02-12 12:04:06,009 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:06,034 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum/_temporary
2015-02-12 12:04:06,073 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:06,112 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:06,113 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/sum
2015-02-12 12:04:06,144 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:06,145 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:06,154 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:06,166 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:06,167 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:06,173 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs] starting
2015-02-12 12:04:06,174 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt"]
2015-02-12 12:04:06,174 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:06,174 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs]  sink: Hfs["TextDelimited[['char', 'sum']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum"]
2015-02-12 12:04:06,174 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs]  parallel execution is enabled: false
2015-02-12 12:04:06,174 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs]  starting jobs: 1
2015-02-12 12:04:06,174 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs]  allocating threads: 1
2015-02-12 12:04:06,174 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum-lhs+sum-rhs] starting step: (1/1) ...yhelpersplatform/mergesum
2015-02-12 12:04:06,175 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:06,176 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:06,206 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:06,207 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:06,233 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:2
2015-02-12 12:04:06,253 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local514690992_0016
2015-02-12 12:04:06,335 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:06,336 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:06,336 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum-lhs+sum-rhs] submitted hadoop job: job_local514690992_0016
2015-02-12 12:04:06,336 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:06,336 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum-lhs+sum-rhs] tracking url: http://localhost:8080/
2015-02-12 12:04:06,338 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:06,339 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local514690992_0016_m_000000_0
2015-02-12 12:04:06,340 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:06,340 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:06,341 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:04:06,341 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1189e8c4
2015-02-12 12:04:06,342 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:06,370 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:06,370 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:06,370 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:06,370 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:06,371 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:06,371 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:06,371 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:06,372 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:06,379 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:06,379 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2015-02-12 12:04:06,380 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:06,381 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:06,381 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 156, total: 569, max: 672
2015-02-12 12:04:06,381 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:06,382 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 156, total: 569, max: 672
2015-02-12 12:04:06,382 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:04:06,382 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 156, total: 569, max: 672
2015-02-12 12:04:06,382 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:04:06,383 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 156, total: 569, max: 672
2015-02-12 12:04:06,384 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:06,384 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:06,384 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:06,384 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-12 12:04:06,384 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:04:06,386 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:06,388 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local514690992_0016_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:06,389 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:04:06,389 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local514690992_0016_m_000000_0' done.
2015-02-12 12:04:06,390 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local514690992_0016_m_000000_0
2015-02-12 12:04:06,390 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local514690992_0016_m_000001_0
2015-02-12 12:04:06,391 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:06,391 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:06,392 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt
2015-02-12 12:04:06,392 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@477500c0
2015-02-12 12:04:06,393 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:06,410 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:06,410 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:06,410 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:06,411 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:06,411 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:06,411 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:06,412 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:06,412 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:06,420 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt"]
2015-02-12 12:04:06,420 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2015-02-12 12:04:06,421 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:06,424 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:06,424 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 55, total: 569, max: 672
2015-02-12 12:04:06,425 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:06,425 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 54, total: 569, max: 672
2015-02-12 12:04:06,426 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:04:06,426 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 54, total: 569, max: 672
2015-02-12 12:04:06,426 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:04:06,427 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 54, total: 569, max: 672
2015-02-12 12:04:06,428 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:06,428 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:06,428 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:06,428 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-12 12:04:06,428 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:04:06,430 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:06,431 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local514690992_0016_m_000001_0 is done. And is in the process of committing
2015-02-12 12:04:06,433 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt:0+51
2015-02-12 12:04:06,433 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local514690992_0016_m_000001_0' done.
2015-02-12 12:04:06,433 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local514690992_0016_m_000001_0
2015-02-12 12:04:06,433 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:06,434 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:06,434 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local514690992_0016_r_000000_0
2015-02-12 12:04:06,435 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:06,435 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:06,435 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@4f86f5f
2015-02-12 12:04:06,436 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:06,437 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local514690992_0016_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:06,438 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#14 about to shuffle output of map attempt_local514690992_0016_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-12 12:04:06,439 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local514690992_0016_m_000000_0
2015-02-12 12:04:06,439 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-12 12:04:06,440 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#14 about to shuffle output of map attempt_local514690992_0016_m_000001_0 decomp: 119 len: 123 to MEMORY
2015-02-12 12:04:06,441 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local514690992_0016_m_000001_0
2015-02-12 12:04:06,441 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 2, commitMemory -> 119, usedMemory ->238
2015-02-12 12:04:06,441 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:06,442 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:04:06,442 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:06,443 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 2 sorted segments
2015-02-12 12:04:06,443 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 2 segments left of total size: 220 bytes
2015-02-12 12:04:06,444 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 2 segments, 238 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:06,445 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 240 bytes from disk
2015-02-12 12:04:06,445 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:06,445 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:06,445 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 227 bytes
2015-02-12 12:04:06,446 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:04:06,446 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:06,446 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:06,453 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2015-02-12 12:04:06,453 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum"]
2015-02-12 12:04:06,466 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local514690992_0016_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:06,467 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:04:06,467 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local514690992_0016_r_000000_0 is allowed to commit now
2015-02-12 12:04:06,468 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local514690992_0016_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum/_temporary/0/task_local514690992_0016_r_000000
2015-02-12 12:04:06,470 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:06,470 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local514690992_0016_r_000000_0' done.
2015-02-12 12:04:06,470 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local514690992_0016_r_000000_0
2015-02-12 12:04:06,471 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:06,494 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum/_temporary
2015-02-12 12:04:06,530 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:06,577 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:06,579 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/mergesum
2015-02-12 12:04:06,613 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:06,613 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:06,619 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:06,621 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:06,621 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:06,627 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-12 12:04:06,628 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt"]
2015-02-12 12:04:06,628 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  sink: Hfs["TextLine[['num']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow"]
2015-02-12 12:04:06,628 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  parallel execution is enabled: false
2015-02-12 12:04:06,628 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  starting jobs: 1
2015-02-12 12:04:06,628 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  allocating threads: 1
2015-02-12 12:04:06,628 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...ersplatform/discardnarrow
2015-02-12 12:04:06,629 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:06,630 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:06,659 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:06,689 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:06,710 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local1986529035_0017
2015-02-12 12:04:06,797 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:06,798 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:06,798 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:06,798 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1986529035_0017
2015-02-12 12:04:06,798 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-12 12:04:06,800 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:06,800 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1986529035_0017_m_000000_0
2015-02-12 12:04:06,802 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:06,802 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:06,803 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt
2015-02-12 12:04:06,803 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@31a1743
2015-02-12 12:04:06,804 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 0
2015-02-12 12:04:06,811 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:06,811 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:06,815 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt"]
2015-02-12 12:04:06,815 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['num']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow"]
2015-02-12 12:04:06,817 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:06,817 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1986529035_0017_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:06,819 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:06,819 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local1986529035_0017_m_000000_0 is allowed to commit now
2015-02-12 12:04:06,819 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1986529035_0017_m_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow/_temporary/0/task_local1986529035_0017_m_000000
2015-02-12 12:04:06,820 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt:0+19
2015-02-12 12:04:06,821 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1986529035_0017_m_000000_0' done.
2015-02-12 12:04:06,821 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1986529035_0017_m_000000_0
2015-02-12 12:04:06,821 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:06,833 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow/_temporary
2015-02-12 12:04:06,873 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:06,874 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/discardnarrow
2015-02-12 12:04:06,893 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:06,893 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:06,899 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:06,908 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:06,909 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:06,913 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-12 12:04:06,914 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt"]
2015-02-12 12:04:06,914 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  sink: Hfs["TextLine[['line']->['item', 'element']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename"]
2015-02-12 12:04:06,914 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  parallel execution is enabled: false
2015-02-12 12:04:06,914 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  starting jobs: 1
2015-02-12 12:04:06,914 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  allocating threads: 1
2015-02-12 12:04:06,915 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...blyhelpersplatform/rename
2015-02-12 12:04:06,915 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:06,916 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:06,946 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:06,971 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:06,988 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local805703128_0018
2015-02-12 12:04:07,073 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:07,073 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:07,074 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:07,074 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local805703128_0018
2015-02-12 12:04:07,074 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-12 12:04:07,076 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:07,076 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local805703128_0018_m_000000_0
2015-02-12 12:04:07,077 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:07,077 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:07,078 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt
2015-02-12 12:04:07,078 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@af62cdf
2015-02-12 12:04:07,079 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 0
2015-02-12 12:04:07,087 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:07,087 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:07,091 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt"]
2015-02-12 12:04:07,091 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['item', 'element']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename"]
2015-02-12 12:04:07,093 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:07,093 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local805703128_0018_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:07,095 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:07,095 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local805703128_0018_m_000000_0 is allowed to commit now
2015-02-12 12:04:07,096 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local805703128_0018_m_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename/_temporary/0/task_local805703128_0018_m_000000
2015-02-12 12:04:07,103 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt:0+19
2015-02-12 12:04:07,103 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local805703128_0018_m_000000_0' done.
2015-02-12 12:04:07,103 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local805703128_0018_m_000000_0
2015-02-12 12:04:07,103 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:07,140 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename/_temporary
2015-02-12 12:04:07,180 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:07,181 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/rename
2015-02-12 12:04:07,206 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:07,207 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:07,281 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:07,284 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:07,285 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:07,294 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name] starting
2015-02-12 12:04:07,295 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt"]
2015-02-12 12:04:07,295 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:07,295 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true"]
2015-02-12 12:04:07,295 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  parallel execution is enabled: false
2015-02-12 12:04:07,295 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  starting jobs: 1
2015-02-12 12:04:07,295 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  allocating threads: 1
2015-02-12 12:04:07,296 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] starting step: (1/1) ...splatform/multimerge+true
2015-02-12 12:04:07,296 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:07,297 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:07,327 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:07,328 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:07,353 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:2
2015-02-12 12:04:07,370 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local937876075_0019
2015-02-12 12:04:07,490 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:07,490 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:07,491 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:07,490 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] submitted hadoop job: job_local937876075_0019
2015-02-12 12:04:07,491 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] tracking url: http://localhost:8080/
2015-02-12 12:04:07,504 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:07,504 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local937876075_0019_m_000000_0
2015-02-12 12:04:07,505 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:07,505 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:07,506 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:04:07,506 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@327e5aba
2015-02-12 12:04:07,507 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:07,537 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:07,537 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:07,537 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:07,537 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:07,537 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:07,538 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:07,538 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:07,538 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:07,548 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:07,548 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2015-02-12 12:04:07,548 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:07,550 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:07,550 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 155, total: 570, max: 672
2015-02-12 12:04:07,551 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:07,551 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 155, total: 570, max: 672
2015-02-12 12:04:07,551 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:04:07,552 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 155, total: 570, max: 672
2015-02-12 12:04:07,552 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:04:07,552 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 155, total: 570, max: 672
2015-02-12 12:04:07,553 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:07,553 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:07,554 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:07,554 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2015-02-12 12:04:07,554 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:04:07,555 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:07,557 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local937876075_0019_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:07,558 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:04:07,559 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local937876075_0019_m_000000_0' done.
2015-02-12 12:04:07,559 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local937876075_0019_m_000000_0
2015-02-12 12:04:07,559 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local937876075_0019_m_000001_0
2015-02-12 12:04:07,560 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:07,560 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:07,561 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt
2015-02-12 12:04:07,561 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3e25ea83
2015-02-12 12:04:07,562 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:07,576 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:07,576 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:07,576 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:07,576 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:07,576 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:07,578 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:07,578 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:07,578 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:07,587 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt"]
2015-02-12 12:04:07,587 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2015-02-12 12:04:07,587 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:07,591 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:07,591 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 53, total: 570, max: 672
2015-02-12 12:04:07,592 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:07,592 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 53, total: 570, max: 672
2015-02-12 12:04:07,593 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:04:07,593 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 53, total: 570, max: 672
2015-02-12 12:04:07,593 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:04:07,593 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 53, total: 570, max: 672
2015-02-12 12:04:07,594 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:07,595 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:07,595 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:07,595 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2015-02-12 12:04:07,595 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:04:07,597 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:07,599 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local937876075_0019_m_000001_0 is done. And is in the process of committing
2015-02-12 12:04:07,600 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt:0+51
2015-02-12 12:04:07,600 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local937876075_0019_m_000001_0' done.
2015-02-12 12:04:07,601 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local937876075_0019_m_000001_0
2015-02-12 12:04:07,601 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:07,601 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:07,601 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local937876075_0019_r_000000_0
2015-02-12 12:04:07,602 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:07,602 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:07,603 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@328b2f19
2015-02-12 12:04:07,604 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:07,604 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local937876075_0019_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:07,606 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#15 about to shuffle output of map attempt_local937876075_0019_m_000000_0 decomp: 236 len: 240 to MEMORY
2015-02-12 12:04:07,606 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local937876075_0019_m_000000_0
2015-02-12 12:04:07,607 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->236
2015-02-12 12:04:07,608 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#15 about to shuffle output of map attempt_local937876075_0019_m_000001_0 decomp: 236 len: 240 to MEMORY
2015-02-12 12:04:07,608 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local937876075_0019_m_000001_0
2015-02-12 12:04:07,608 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 2, commitMemory -> 236, usedMemory ->472
2015-02-12 12:04:07,608 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:07,609 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:04:07,609 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:07,611 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 2 sorted segments
2015-02-12 12:04:07,611 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 2 segments left of total size: 454 bytes
2015-02-12 12:04:07,612 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 2 segments, 472 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:07,613 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 474 bytes from disk
2015-02-12 12:04:07,613 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:07,613 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:07,613 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 461 bytes
2015-02-12 12:04:07,614 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:04:07,614 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:07,615 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:07,624 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(name)[by:[{1}:'char']]
2015-02-12 12:04:07,624 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true"]
2015-02-12 12:04:07,636 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local937876075_0019_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:07,637 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:04:07,637 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local937876075_0019_r_000000_0 is allowed to commit now
2015-02-12 12:04:07,638 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local937876075_0019_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true/_temporary/0/task_local937876075_0019_r_000000
2015-02-12 12:04:07,639 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:07,639 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local937876075_0019_r_000000_0' done.
2015-02-12 12:04:07,639 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local937876075_0019_r_000000_0
2015-02-12 12:04:07,640 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:07,665 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true/_temporary
2015-02-12 12:04:07,700 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:07,742 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:07,743 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/multimerge+true
2015-02-12 12:04:07,772 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:07,772 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:07,803 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:07,805 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:07,812 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:07,820 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [second] starting
2015-02-12 12:04:07,820 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [second]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:07,820 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [second]  sink: Hfs["TextDelimited[['char', 'count']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount"]
2015-02-12 12:04:07,820 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [second]  parallel execution is enabled: false
2015-02-12 12:04:07,820 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [second]  starting jobs: 2
2015-02-12 12:04:07,820 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [second]  allocating threads: 1
2015-02-12 12:04:07,821 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] starting step: (1/2)
2015-02-12 12:04:07,821 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:07,822 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:07,851 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:07,877 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:07,898 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local396840954_0020
2015-02-12 12:04:07,972 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:07,972 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:07,973 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:07,974 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] submitted hadoop job: job_local396840954_0020
2015-02-12 12:04:07,975 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local396840954_0020_m_000000_0
2015-02-12 12:04:07,975 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:07,975 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] tracking url: http://localhost:8080/
2015-02-12 12:04:07,976 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:07,976 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:07,977 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:04:07,977 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2ee6a5eb
2015-02-12 12:04:07,978 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:07,993 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:07,993 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:07,993 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:07,993 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:07,993 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:07,994 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:07,994 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:07,994 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:08,001 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:08,001 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(first)[by:[{1}:'char']]
2015-02-12 12:04:08,002 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:08,005 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:08,005 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 112, total: 570, max: 672
2015-02-12 12:04:08,006 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:08,006 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 112, total: 570, max: 672
2015-02-12 12:04:08,006 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:04:08,006 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 112, total: 570, max: 672
2015-02-12 12:04:08,007 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:04:08,007 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 112, total: 570, max: 672
2015-02-12 12:04:08,008 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:08,008 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:08,008 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:08,008 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-12 12:04:08,008 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:04:08,011 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:08,013 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local396840954_0020_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:08,015 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:04:08,015 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local396840954_0020_m_000000_0' done.
2015-02-12 12:04:08,015 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local396840954_0020_m_000000_0
2015-02-12 12:04:08,015 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:08,016 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:08,016 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local396840954_0020_r_000000_0
2015-02-12 12:04:08,017 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:08,017 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:08,018 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7b8c3169
2015-02-12 12:04:08,018 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:08,019 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local396840954_0020_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:08,020 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#16 about to shuffle output of map attempt_local396840954_0020_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-12 12:04:08,020 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local396840954_0020_m_000000_0
2015-02-12 12:04:08,020 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-12 12:04:08,021 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:08,021 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:08,021 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:08,023 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:08,023 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-12 12:04:08,024 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:08,025 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 123 bytes from disk
2015-02-12 12:04:08,025 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:08,025 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:08,025 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-12 12:04:08,026 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:08,026 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:08,026 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:08,032 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(first)[by:[{1}:'char']]
2015-02-12 12:04:08,032 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][6555200240/first/]
2015-02-12 12:04:08,043 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local396840954_0020_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:08,044 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:08,044 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local396840954_0020_r_000000_0 is allowed to commit now
2015-02-12 12:04:08,045 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local396840954_0020_r_000000_0' to file:/tmp/hadoop-soberoi/6555200240_first_2C2658D7E70745C680758399725F1199/_temporary/0/task_local396840954_0020_r_000000
2015-02-12 12:04:08,046 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:08,046 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local396840954_0020_r_000000_0' done.
2015-02-12 12:04:08,046 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local396840954_0020_r_000000_0
2015-02-12 12:04:08,046 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:08,060 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] starting step: (2/2) ...form/samesourcemergecount
2015-02-12 12:04:08,060 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:08,061 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:08,110 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:08,137 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:08,157 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:08,157 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:08,163 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local211712279_0021
2015-02-12 12:04:08,300 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:08,301 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:08,301 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] submitted hadoop job: job_local211712279_0021
2015-02-12 12:04:08,301 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:08,301 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] tracking url: http://localhost:8080/
2015-02-12 12:04:08,303 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:08,303 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local211712279_0021_m_000000_0
2015-02-12 12:04:08,305 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:08,305 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:08,306 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-soberoi/6555200240_first_2C2658D7E70745C680758399725F1199/part-00000
2015-02-12 12:04:08,306 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@48fe0cf8
2015-02-12 12:04:08,307 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:08,325 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:08,325 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:08,325 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:08,326 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:08,326 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:08,327 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:08,327 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:08,327 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:08,335 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][6555200240/first/]
2015-02-12 12:04:08,335 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(second)[by:[{1}:'char']]
2015-02-12 12:04:08,335 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:08,337 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:08,337 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 156, total: 570, max: 672
2015-02-12 12:04:08,337 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:08,337 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 156, total: 570, max: 672
2015-02-12 12:04:08,339 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:08,339 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:08,339 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:08,339 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 55; bufvoid = 104857600
2015-02-12 12:04:08,339 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-12 12:04:08,341 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:08,342 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local211712279_0021_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:08,344 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-soberoi/6555200240_first_2C2658D7E70745C680758399725F1199/part-00000:0+160
2015-02-12 12:04:08,344 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local211712279_0021_m_000000_0' done.
2015-02-12 12:04:08,344 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local211712279_0021_m_000000_0
2015-02-12 12:04:08,344 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:08,345 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:08,345 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local211712279_0021_r_000000_0
2015-02-12 12:04:08,346 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:08,346 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:08,347 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@302c3e0f
2015-02-12 12:04:08,347 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:08,348 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local211712279_0021_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:08,349 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#17 about to shuffle output of map attempt_local211712279_0021_m_000000_0 decomp: 67 len: 71 to MEMORY
2015-02-12 12:04:08,349 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 67 bytes from map-output for attempt_local211712279_0021_m_000000_0
2015-02-12 12:04:08,349 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 67, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->67
2015-02-12 12:04:08,350 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:08,351 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:08,351 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:08,352 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:08,352 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-12 12:04:08,353 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 67 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:08,353 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 71 bytes from disk
2015-02-12 12:04:08,353 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:08,353 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:08,354 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-12 12:04:08,354 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:08,354 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:08,355 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:08,362 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(second)[by:[{1}:'char']]
2015-02-12 12:04:08,362 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount"]
2015-02-12 12:04:08,373 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local211712279_0021_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:08,374 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:08,375 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local211712279_0021_r_000000_0 is allowed to commit now
2015-02-12 12:04:08,376 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local211712279_0021_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount/_temporary/0/task_local211712279_0021_r_000000
2015-02-12 12:04:08,377 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:08,377 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local211712279_0021_r_000000_0' done.
2015-02-12 12:04:08,377 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local211712279_0021_r_000000_0
2015-02-12 12:04:08,377 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:08,415 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount/_temporary
2015-02-12 12:04:08,449 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:08,489 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:08,491 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount
2015-02-12 12:04:08,512 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:08,513 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:08,518 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:08,520 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:08,521 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:08,526 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average] starting
2015-02-12 12:04:08,526 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:08,526 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  sink: Hfs["TextDelimited[['char', 'average']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average"]
2015-02-12 12:04:08,526 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  parallel execution is enabled: false
2015-02-12 12:04:08,526 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  starting jobs: 1
2015-02-12 12:04:08,526 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  allocating threads: 1
2015-02-12 12:04:08,527 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] starting step: (1/1) ...lyhelpersplatform/average
2015-02-12 12:04:08,527 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:08,528 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:08,558 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:08,583 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:08,601 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local1389250731_0022
2015-02-12 12:04:08,674 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:08,674 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:08,674 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:08,674 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] submitted hadoop job: job_local1389250731_0022
2015-02-12 12:04:08,676 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1389250731_0022_m_000000_0
2015-02-12 12:04:08,676 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:08,676 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] tracking url: http://localhost:8080/
2015-02-12 12:04:08,678 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:08,678 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:08,679 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:04:08,679 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@421d75a8
2015-02-12 12:04:08,680 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:08,708 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:08,708 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:08,708 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:08,708 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:08,708 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:08,709 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:08,709 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:08,709 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:08,714 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:08,715 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average)[by:[{1}:'char']]
2015-02-12 12:04:08,715 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:08,716 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:08,717 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 159, total: 571, max: 672
2015-02-12 12:04:08,717 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:08,717 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 159, total: 571, max: 672
2015-02-12 12:04:08,718 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:04:08,718 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 159, total: 571, max: 672
2015-02-12 12:04:08,718 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:04:08,718 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 158, total: 571, max: 672
2015-02-12 12:04:08,720 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:08,720 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:08,720 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:08,720 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2015-02-12 12:04:08,720 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:04:08,722 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:08,723 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1389250731_0022_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:08,725 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:04:08,725 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1389250731_0022_m_000000_0' done.
2015-02-12 12:04:08,725 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1389250731_0022_m_000000_0
2015-02-12 12:04:08,725 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:08,726 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:08,726 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1389250731_0022_r_000000_0
2015-02-12 12:04:08,727 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:08,727 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:08,727 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@30b6c3c3
2015-02-12 12:04:08,728 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:08,728 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1389250731_0022_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:08,729 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#18 about to shuffle output of map attempt_local1389250731_0022_m_000000_0 decomp: 200 len: 204 to MEMORY
2015-02-12 12:04:08,730 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local1389250731_0022_m_000000_0
2015-02-12 12:04:08,730 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->200
2015-02-12 12:04:08,730 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:08,731 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:08,731 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:08,743 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:08,743 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2015-02-12 12:04:08,744 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 200 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:08,744 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 204 bytes from disk
2015-02-12 12:04:08,744 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:08,744 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:08,745 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2015-02-12 12:04:08,745 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:08,745 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:08,745 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:08,751 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(average)[by:[{1}:'char']]
2015-02-12 12:04:08,751 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average"]
2015-02-12 12:04:08,761 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1389250731_0022_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:08,762 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:08,762 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local1389250731_0022_r_000000_0 is allowed to commit now
2015-02-12 12:04:08,763 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1389250731_0022_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average/_temporary/0/task_local1389250731_0022_r_000000
2015-02-12 12:04:08,764 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:08,764 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1389250731_0022_r_000000_0' done.
2015-02-12 12:04:08,765 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1389250731_0022_r_000000_0
2015-02-12 12:04:08,765 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:08,805 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average/_temporary
2015-02-12 12:04:08,847 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:08,895 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:08,896 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/average
2015-02-12 12:04:08,919 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:08,941 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:08,946 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:08,948 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:08,948 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:08,954 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting
2015-02-12 12:04:08,954 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:08,954 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  sink: Hfs["TextDelimited[['char', 'max']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety"]
2015-02-12 12:04:08,954 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  parallel execution is enabled: false
2015-02-12 12:04:08,954 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  starting jobs: 1
2015-02-12 12:04:08,954 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  allocating threads: 1
2015-02-12 12:04:08,955 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...splatform/maxbynullsafety
2015-02-12 12:04:08,955 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:08,956 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:08,985 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:09,011 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:09,030 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local1163634463_0023
2015-02-12 12:04:09,259 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:09,259 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:09,259 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:09,259 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local1163634463_0023
2015-02-12 12:04:09,260 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2015-02-12 12:04:09,261 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:09,261 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1163634463_0023_m_000000_0
2015-02-12 12:04:09,262 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:09,262 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:09,263 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:04:09,263 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@bda23fa
2015-02-12 12:04:09,264 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:09,293 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:09,293 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:09,294 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:09,294 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:09,294 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:09,295 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:09,295 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:09,295 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:09,301 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:09,301 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'char']]
2015-02-12 12:04:09,301 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:09,303 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:09,303 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 159, total: 571, max: 672
2015-02-12 12:04:09,303 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:09,304 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 159, total: 571, max: 672
2015-02-12 12:04:09,304 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:04:09,304 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 158, total: 571, max: 672
2015-02-12 12:04:09,305 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:04:09,305 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 158, total: 571, max: 672
2015-02-12 12:04:09,306 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:09,307 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:09,307 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:09,307 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 98; bufvoid = 104857600
2015-02-12 12:04:09,307 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:04:09,309 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:09,311 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1163634463_0023_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:09,313 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:04:09,313 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1163634463_0023_m_000000_0' done.
2015-02-12 12:04:09,313 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1163634463_0023_m_000000_0
2015-02-12 12:04:09,314 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:09,314 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:09,314 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1163634463_0023_r_000000_0
2015-02-12 12:04:09,315 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:09,315 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:09,315 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5d4ce711
2015-02-12 12:04:09,316 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:09,317 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1163634463_0023_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:09,318 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#19 about to shuffle output of map attempt_local1163634463_0023_m_000000_0 decomp: 118 len: 122 to MEMORY
2015-02-12 12:04:09,318 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 118 bytes from map-output for attempt_local1163634463_0023_m_000000_0
2015-02-12 12:04:09,318 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 118, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->118
2015-02-12 12:04:09,319 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:09,320 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:09,320 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:09,321 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:09,321 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 109 bytes
2015-02-12 12:04:09,322 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 118 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:09,322 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 122 bytes from disk
2015-02-12 12:04:09,322 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:09,322 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:09,323 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 109 bytes
2015-02-12 12:04:09,323 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:09,323 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:09,323 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:09,329 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'char']]
2015-02-12 12:04:09,329 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'max']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety"]
2015-02-12 12:04:09,340 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1163634463_0023_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:09,342 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:09,342 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local1163634463_0023_r_000000_0 is allowed to commit now
2015-02-12 12:04:09,343 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1163634463_0023_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety/_temporary/0/task_local1163634463_0023_r_000000
2015-02-12 12:04:09,344 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:09,345 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1163634463_0023_r_000000_0' done.
2015-02-12 12:04:09,345 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1163634463_0023_r_000000_0
2015-02-12 12:04:09,345 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:09,359 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety/_temporary
2015-02-12 12:04:09,396 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:09,471 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:09,473 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety
2015-02-12 12:04:09,499 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:09,500 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:09,506 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:09,510 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:09,510 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:09,560 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] starting
2015-02-12 12:04:09,560 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:09,560 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  sink: Hfs["TextDelimited[['char', 'sum']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls"]
2015-02-12 12:04:09,561 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  parallel execution is enabled: false
2015-02-12 12:04:09,561 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  starting jobs: 1
2015-02-12 12:04:09,561 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  allocating threads: 1
2015-02-12 12:04:09,561 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] starting step: (1/1) ...yhelpersplatform/sumnulls
2015-02-12 12:04:09,562 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:09,563 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:09,593 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:09,619 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:09,638 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local456833703_0024
2015-02-12 12:04:09,746 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:09,746 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:09,747 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:09,747 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] submitted hadoop job: job_local456833703_0024
2015-02-12 12:04:09,747 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] tracking url: http://localhost:8080/
2015-02-12 12:04:09,749 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:09,749 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local456833703_0024_m_000000_0
2015-02-12 12:04:09,750 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:09,750 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:09,751 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:04:09,752 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@787935a1
2015-02-12 12:04:09,753 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:09,771 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:09,771 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:09,772 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:09,773 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:09,773 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:09,775 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:09,775 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:09,776 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:09,781 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:09,781 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum)[by:[{1}:'char']]
2015-02-12 12:04:09,782 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:09,789 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:09,789 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 159, total: 572, max: 672
2015-02-12 12:04:09,790 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:09,790 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 159, total: 572, max: 672
2015-02-12 12:04:09,791 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:04:09,791 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 159, total: 572, max: 672
2015-02-12 12:04:09,791 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:04:09,792 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 159, total: 572, max: 672
2015-02-12 12:04:09,793 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:09,793 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:09,793 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:09,793 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 96; bufvoid = 104857600
2015-02-12 12:04:09,793 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:04:09,795 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:09,796 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local456833703_0024_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:09,798 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:04:09,798 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local456833703_0024_m_000000_0' done.
2015-02-12 12:04:09,798 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local456833703_0024_m_000000_0
2015-02-12 12:04:09,798 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:09,799 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:09,799 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local456833703_0024_r_000000_0
2015-02-12 12:04:09,800 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:09,800 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:09,800 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@18a062f6
2015-02-12 12:04:09,801 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:09,801 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local456833703_0024_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:09,802 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#20 about to shuffle output of map attempt_local456833703_0024_m_000000_0 decomp: 116 len: 120 to MEMORY
2015-02-12 12:04:09,803 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 116 bytes from map-output for attempt_local456833703_0024_m_000000_0
2015-02-12 12:04:09,803 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 116, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->116
2015-02-12 12:04:09,803 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:09,804 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:09,804 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:09,805 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:09,805 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 107 bytes
2015-02-12 12:04:09,806 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 116 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:09,806 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 120 bytes from disk
2015-02-12 12:04:09,806 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:09,806 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:09,806 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 107 bytes
2015-02-12 12:04:09,807 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:09,807 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:09,807 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:09,812 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(sum)[by:[{1}:'char']]
2015-02-12 12:04:09,812 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls"]
2015-02-12 12:04:09,822 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local456833703_0024_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:09,823 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:09,823 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local456833703_0024_r_000000_0 is allowed to commit now
2015-02-12 12:04:09,824 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local456833703_0024_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls/_temporary/0/task_local456833703_0024_r_000000
2015-02-12 12:04:09,825 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:09,825 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local456833703_0024_r_000000_0' done.
2015-02-12 12:04:09,825 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local456833703_0024_r_000000_0
2015-02-12 12:04:09,825 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:09,846 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls/_temporary
2015-02-12 12:04:09,879 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:09,914 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:09,915 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/sumnulls
2015-02-12 12:04:09,935 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:09,936 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:09,941 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:09,943 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:09,943 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:09,948 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting
2015-02-12 12:04:09,948 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:09,948 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  sink: Hfs["TextDelimited[['count']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall"]
2015-02-12 12:04:09,948 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  parallel execution is enabled: false
2015-02-12 12:04:09,948 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  starting jobs: 1
2015-02-12 12:04:09,949 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  allocating threads: 1
2015-02-12 12:04:09,949 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...yhelpersplatform/countall
2015-02-12 12:04:09,950 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:09,951 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:09,980 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:10,005 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:10,024 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local419681603_0025
2015-02-12 12:04:10,090 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:10,090 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:10,090 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local419681603_0025
2015-02-12 12:04:10,090 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:10,090 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-12 12:04:10,092 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:10,092 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local419681603_0025_m_000000_0
2015-02-12 12:04:10,093 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:10,093 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:10,094 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:04:10,094 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@8e8ec29
2015-02-12 12:04:10,094 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:10,111 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:10,111 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:10,111 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:10,112 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:10,112 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:10,113 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:10,113 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:10,113 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:10,117 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:10,117 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{?}:NONE]]
2015-02-12 12:04:10,117 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:10,119 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:10,119 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:10,119 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:10,119 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 4; bufvoid = 104857600
2015-02-12 12:04:10,120 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214396(104857584); length = 1/6553600
2015-02-12 12:04:10,121 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:10,122 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local419681603_0025_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:10,123 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:04:10,124 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local419681603_0025_m_000000_0' done.
2015-02-12 12:04:10,124 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local419681603_0025_m_000000_0
2015-02-12 12:04:10,124 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:10,124 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:10,124 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local419681603_0025_r_000000_0
2015-02-12 12:04:10,125 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:10,126 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:10,126 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@33dba6d1
2015-02-12 12:04:10,126 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:10,127 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local419681603_0025_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:10,128 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#21 about to shuffle output of map attempt_local419681603_0025_m_000000_0 decomp: 8 len: 12 to MEMORY
2015-02-12 12:04:10,128 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 8 bytes from map-output for attempt_local419681603_0025_m_000000_0
2015-02-12 12:04:10,128 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 8, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->8
2015-02-12 12:04:10,129 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:10,130 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:10,130 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:10,131 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:10,131 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 5 bytes
2015-02-12 12:04:10,132 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 8 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:10,132 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 12 bytes from disk
2015-02-12 12:04:10,132 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:10,133 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:10,133 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 5 bytes
2015-02-12 12:04:10,133 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:10,134 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:10,134 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:10,139 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{?}:NONE]]
2015-02-12 12:04:10,140 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['count']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall"]
2015-02-12 12:04:10,147 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local419681603_0025_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:10,148 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:10,148 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local419681603_0025_r_000000_0 is allowed to commit now
2015-02-12 12:04:10,149 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local419681603_0025_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall/_temporary/0/task_local419681603_0025_r_000000
2015-02-12 12:04:10,149 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:10,150 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local419681603_0025_r_000000_0' done.
2015-02-12 12:04:10,150 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local419681603_0025_r_000000_0
2015-02-12 12:04:10,150 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:10,168 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall/_temporary
2015-02-12 12:04:10,193 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:10,215 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:10,216 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/countall
2015-02-12 12:04:10,229 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:10,229 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:10,234 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:10,235 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:10,236 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:10,240 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name] starting
2015-02-12 12:04:10,240 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:10,241 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average', 'average2', 'first']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi"]
2015-02-12 12:04:10,241 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  parallel execution is enabled: false
2015-02-12 12:04:10,241 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  starting jobs: 1
2015-02-12 12:04:10,241 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  allocating threads: 1
2015-02-12 12:04:10,241 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] starting step: (1/1) ...mblyhelpersplatform/multi
2015-02-12 12:04:10,242 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:10,242 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:10,259 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:10,274 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:10,286 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local1735295780_0026
2015-02-12 12:04:10,331 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:10,331 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] submitted hadoop job: job_local1735295780_0026
2015-02-12 12:04:10,331 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:10,332 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:10,331 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] tracking url: http://localhost:8080/
2015-02-12 12:04:10,333 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:10,333 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1735295780_0026_m_000000_0
2015-02-12 12:04:10,334 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:10,334 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:10,335 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:04:10,335 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6af5679
2015-02-12 12:04:10,336 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:10,351 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:10,351 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:10,351 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:10,351 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:10,351 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:10,352 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:10,352 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:10,353 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:10,357 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:10,357 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2015-02-12 12:04:10,358 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:10,358 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:10,359 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 160, total: 573, max: 672
2015-02-12 12:04:10,359 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:10,359 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 160, total: 573, max: 672
2015-02-12 12:04:10,360 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:04:10,360 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 160, total: 573, max: 672
2015-02-12 12:04:10,360 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:04:10,360 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 160, total: 573, max: 672
2015-02-12 12:04:10,361 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:10,361 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:10,361 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:10,361 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 432; bufvoid = 104857600
2015-02-12 12:04:10,361 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:04:10,362 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:10,363 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1735295780_0026_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:10,364 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:04:10,364 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1735295780_0026_m_000000_0' done.
2015-02-12 12:04:10,364 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1735295780_0026_m_000000_0
2015-02-12 12:04:10,365 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:10,365 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:10,365 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1735295780_0026_r_000000_0
2015-02-12 12:04:10,366 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:10,366 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:10,366 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@72a9b899
2015-02-12 12:04:10,366 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:10,367 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1735295780_0026_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:10,368 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#22 about to shuffle output of map attempt_local1735295780_0026_m_000000_0 decomp: 452 len: 456 to MEMORY
2015-02-12 12:04:10,368 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 452 bytes from map-output for attempt_local1735295780_0026_m_000000_0
2015-02-12 12:04:10,368 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 452, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->452
2015-02-12 12:04:10,369 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:10,369 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:10,369 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:10,371 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:10,371 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 436 bytes
2015-02-12 12:04:10,371 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 452 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:10,372 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 456 bytes from disk
2015-02-12 12:04:10,372 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:10,372 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:10,372 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 436 bytes
2015-02-12 12:04:10,373 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:10,373 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:10,373 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:10,380 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(name)[by:[{1}:'char']]
2015-02-12 12:04:10,380 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average', 'average2', 'first']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi"]
2015-02-12 12:04:10,387 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1735295780_0026_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:10,388 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:10,388 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local1735295780_0026_r_000000_0 is allowed to commit now
2015-02-12 12:04:10,388 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1735295780_0026_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi/_temporary/0/task_local1735295780_0026_r_000000
2015-02-12 12:04:10,389 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:10,389 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1735295780_0026_r_000000_0' done.
2015-02-12 12:04:10,389 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1735295780_0026_r_000000_0
2015-02-12 12:04:10,389 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:10,404 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi/_temporary
2015-02-12 12:04:10,426 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:10,451 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:10,452 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/multi
2015-02-12 12:04:10,464 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:10,464 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:10,467 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:10,468 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:10,468 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:10,471 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-12 12:04:10,472 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt"]
2015-02-12 12:04:10,472 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  sink: Hfs["TextLine[['num']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow"]
2015-02-12 12:04:10,472 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  parallel execution is enabled: false
2015-02-12 12:04:10,472 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  starting jobs: 1
2015-02-12 12:04:10,472 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  allocating threads: 1
2015-02-12 12:04:10,473 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...persplatform/retainnarrow
2015-02-12 12:04:10,473 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:10,474 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:10,491 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:10,506 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:10,518 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local752516465_0027
2015-02-12 12:04:10,559 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:10,560 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:10,560 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:10,560 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local752516465_0027
2015-02-12 12:04:10,560 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-12 12:04:10,561 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:10,561 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local752516465_0027_m_000000_0
2015-02-12 12:04:10,562 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:10,563 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:10,563 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt
2015-02-12 12:04:10,563 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@78084d9d
2015-02-12 12:04:10,564 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 0
2015-02-12 12:04:10,568 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:10,569 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:10,571 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt"]
2015-02-12 12:04:10,571 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['num']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow"]
2015-02-12 12:04:10,573 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:10,573 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local752516465_0027_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:10,574 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:10,574 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local752516465_0027_m_000000_0 is allowed to commit now
2015-02-12 12:04:10,575 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local752516465_0027_m_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow/_temporary/0/task_local752516465_0027_m_000000
2015-02-12 12:04:10,576 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt:0+19
2015-02-12 12:04:10,576 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local752516465_0027_m_000000_0' done.
2015-02-12 12:04:10,576 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local752516465_0027_m_000000_0
2015-02-12 12:04:10,576 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:10,585 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow/_temporary
2015-02-12 12:04:10,607 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:10,608 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/retainnarrow
2015-02-12 12:04:10,625 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:10,626 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:10,628 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:10,630 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:10,630 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:10,633 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-12 12:04:10,633 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt"]
2015-02-12 12:04:10,633 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  sink: Hfs["TextLine[['item']->['char', 'item']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow"]
2015-02-12 12:04:10,633 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  parallel execution is enabled: false
2015-02-12 12:04:10,634 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  starting jobs: 1
2015-02-12 12:04:10,634 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  allocating threads: 1
2015-02-12 12:04:10,634 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...persplatform/renamenarrow
2015-02-12 12:04:10,635 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:10,635 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:10,655 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:10,670 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:10,683 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local1119227726_0028
2015-02-12 12:04:10,726 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:10,726 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:10,727 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1119227726_0028
2015-02-12 12:04:10,727 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:10,727 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-12 12:04:10,728 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:10,728 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1119227726_0028_m_000000_0
2015-02-12 12:04:10,729 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:10,729 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:10,730 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt
2015-02-12 12:04:10,730 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@525c726b
2015-02-12 12:04:10,731 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 0
2015-02-12 12:04:10,735 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:10,735 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:10,737 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt"]
2015-02-12 12:04:10,737 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['item']->['char', 'item']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow"]
2015-02-12 12:04:10,739 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:10,740 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1119227726_0028_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:10,741 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:10,741 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local1119227726_0028_m_000000_0 is allowed to commit now
2015-02-12 12:04:10,741 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1119227726_0028_m_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow/_temporary/0/task_local1119227726_0028_m_000000
2015-02-12 12:04:10,742 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt:0+19
2015-02-12 12:04:10,742 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1119227726_0028_m_000000_0' done.
2015-02-12 12:04:10,742 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1119227726_0028_m_000000_0
2015-02-12 12:04:10,743 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:10,765 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow/_temporary
2015-02-12 12:04:10,789 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:10,791 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/renamenarrow
2015-02-12 12:04:10,810 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:10,810 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:10,813 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:10,814 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:10,815 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:10,819 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average] starting
2015-02-12 12:04:10,819 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:10,819 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  sink: Hfs["TextDelimited[['char', 'average']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull"]
2015-02-12 12:04:10,819 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  parallel execution is enabled: false
2015-02-12 12:04:10,819 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  starting jobs: 1
2015-02-12 12:04:10,819 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  allocating threads: 1
2015-02-12 12:04:10,820 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] starting step: (1/1) ...lpersplatform/averagenull
2015-02-12 12:04:10,820 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:10,821 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:10,846 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:10,863 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:10,879 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local527262299_0029
2015-02-12 12:04:10,953 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:10,953 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:10,953 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:10,953 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] submitted hadoop job: job_local527262299_0029
2015-02-12 12:04:10,954 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] tracking url: http://localhost:8080/
2015-02-12 12:04:10,955 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:10,955 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local527262299_0029_m_000000_0
2015-02-12 12:04:10,956 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:10,957 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:10,957 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:04:10,958 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5e7dac7d
2015-02-12 12:04:10,959 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:10,972 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:10,972 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:10,972 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:10,973 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:10,973 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:10,973 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:10,974 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:10,974 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:10,979 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:10,979 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average)[by:[{1}:'char']]
2015-02-12 12:04:10,980 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:10,983 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:10,983 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 126, total: 573, max: 672
2015-02-12 12:04:10,983 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:10,984 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 126, total: 573, max: 672
2015-02-12 12:04:10,984 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:04:10,984 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 126, total: 573, max: 672
2015-02-12 12:04:10,985 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:04:10,985 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 126, total: 573, max: 672
2015-02-12 12:04:10,986 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:10,986 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:10,986 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:10,986 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2015-02-12 12:04:10,986 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:04:10,988 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:10,989 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local527262299_0029_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:10,991 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:04:10,991 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local527262299_0029_m_000000_0' done.
2015-02-12 12:04:10,991 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local527262299_0029_m_000000_0
2015-02-12 12:04:10,991 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:10,992 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:10,992 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local527262299_0029_r_000000_0
2015-02-12 12:04:10,993 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:10,993 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:10,993 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@6d33fa6d
2015-02-12 12:04:10,994 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:10,994 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local527262299_0029_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:10,995 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#23 about to shuffle output of map attempt_local527262299_0029_m_000000_0 decomp: 200 len: 204 to MEMORY
2015-02-12 12:04:10,996 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local527262299_0029_m_000000_0
2015-02-12 12:04:10,996 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->200
2015-02-12 12:04:10,996 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:10,997 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:10,997 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:10,998 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:10,998 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2015-02-12 12:04:10,999 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 200 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:10,999 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 204 bytes from disk
2015-02-12 12:04:10,999 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:10,999 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:11,000 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2015-02-12 12:04:11,000 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:11,000 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:11,000 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:11,005 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(average)[by:[{1}:'char']]
2015-02-12 12:04:11,006 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull"]
2015-02-12 12:04:11,016 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local527262299_0029_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:11,017 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:11,017 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local527262299_0029_r_000000_0 is allowed to commit now
2015-02-12 12:04:11,017 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local527262299_0029_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull/_temporary/0/task_local527262299_0029_r_000000
2015-02-12 12:04:11,018 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:11,018 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local527262299_0029_r_000000_0' done.
2015-02-12 12:04:11,018 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local527262299_0029_r_000000_0
2015-02-12 12:04:11,019 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:11,038 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull/_temporary
2015-02-12 12:04:11,074 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:11,115 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:11,117 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/averagenull
2015-02-12 12:04:11,135 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:11,135 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:11,140 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:11,142 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:11,143 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:11,147 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] starting
2015-02-12 12:04:11,147 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt"]
2015-02-12 12:04:11,147 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  sink: Hfs["TextLine[['line']->['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce"]
2015-02-12 12:04:11,148 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  parallel execution is enabled: false
2015-02-12 12:04:11,148 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  starting jobs: 1
2015-02-12 12:04:11,148 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  allocating threads: 1
2015-02-12 12:04:11,148 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] starting step: (1/1) ...blyhelpersplatform/coerce
2015-02-12 12:04:11,149 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:11,150 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:11,179 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:11,205 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:11,222 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local1611591678_0030
2015-02-12 12:04:11,275 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:11,275 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:11,276 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:11,276 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] submitted hadoop job: job_local1611591678_0030
2015-02-12 12:04:11,277 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] tracking url: http://localhost:8080/
2015-02-12 12:04:11,278 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:11,278 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1611591678_0030_m_000000_0
2015-02-12 12:04:11,279 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:11,279 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:11,280 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt
2015-02-12 12:04:11,280 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1fff4cac
2015-02-12 12:04:11,281 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 0
2015-02-12 12:04:11,286 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:11,286 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:11,288 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt"]
2015-02-12 12:04:11,289 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce"]
2015-02-12 12:04:11,291 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:11,291 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1611591678_0030_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:11,292 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:11,292 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local1611591678_0030_m_000000_0 is allowed to commit now
2015-02-12 12:04:11,293 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1611591678_0030_m_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce/_temporary/0/task_local1611591678_0030_m_000000
2015-02-12 12:04:11,294 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lower.txt:0+19
2015-02-12 12:04:11,294 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1611591678_0030_m_000000_0' done.
2015-02-12 12:04:11,294 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1611591678_0030_m_000000_0
2015-02-12 12:04:11,294 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:11,313 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce/_temporary
2015-02-12 12:04:11,342 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:11,343 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/coerce
2015-02-12 12:04:11,361 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:11,361 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:11,366 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:11,368 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:11,368 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:11,372 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first] starting
2015-02-12 12:04:11,372 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  source: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs+rhs-cross-rev.txt"]
2015-02-12 12:04:11,372 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  sink: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator"]
2015-02-12 12:04:11,372 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  parallel execution is enabled: false
2015-02-12 12:04:11,372 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  starting jobs: 1
2015-02-12 12:04:11,372 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  allocating threads: 1
2015-02-12 12:04:11,373 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] starting step: (1/1) ...tnfieldswithoutcomparator
2015-02-12 12:04:11,373 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:11,374 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:11,396 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:11,417 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:11,431 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local1736216983_0031
2015-02-12 12:04:11,497 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:11,497 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:11,497 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] submitted hadoop job: job_local1736216983_0031
2015-02-12 12:04:11,497 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] tracking url: http://localhost:8080/
2015-02-12 12:04:11,497 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:11,499 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:11,499 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1736216983_0031_m_000000_0
2015-02-12 12:04:11,500 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:11,500 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:11,501 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs+rhs-cross-rev.txt
2015-02-12 12:04:11,501 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@470a1f17
2015-02-12 12:04:11,502 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:11,515 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:11,515 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:11,515 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:11,515 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:11,515 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:11,516 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:11,516 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:11,516 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:11,520 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs+rhs-cross-rev.txt"]
2015-02-12 12:04:11,521 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(first)[by:[{1}:'num']]
2015-02-12 12:04:11,521 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:11,523 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:11,523 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 142, total: 573, max: 672
2015-02-12 12:04:11,524 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:11,524 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 142, total: 573, max: 672
2015-02-12 12:04:11,525 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:11,525 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:11,525 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:11,526 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 105; bufvoid = 104857600
2015-02-12 12:04:11,526 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-12 12:04:11,527 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:11,528 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1736216983_0031_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:11,529 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs+rhs-cross-rev.txt:0+221
2015-02-12 12:04:11,529 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1736216983_0031_m_000000_0' done.
2015-02-12 12:04:11,530 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1736216983_0031_m_000000_0
2015-02-12 12:04:11,530 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:11,530 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:11,530 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1736216983_0031_r_000000_0
2015-02-12 12:04:11,531 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:11,531 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:11,531 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@225a84be
2015-02-12 12:04:11,532 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:11,532 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1736216983_0031_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:11,534 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#24 about to shuffle output of map attempt_local1736216983_0031_m_000000_0 decomp: 117 len: 121 to MEMORY
2015-02-12 12:04:11,535 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 117 bytes from map-output for attempt_local1736216983_0031_m_000000_0
2015-02-12 12:04:11,535 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 117, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->117
2015-02-12 12:04:11,535 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:11,536 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:11,536 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:11,537 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:11,537 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 108 bytes
2015-02-12 12:04:11,538 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 117 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:11,538 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 121 bytes from disk
2015-02-12 12:04:11,538 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:11,538 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:11,539 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 108 bytes
2015-02-12 12:04:11,539 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:11,539 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:11,539 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:11,543 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(first)[by:[{1}:'num']]
2015-02-12 12:04:11,544 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator"]
2015-02-12 12:04:11,553 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1736216983_0031_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:11,554 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:11,554 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local1736216983_0031_r_000000_0 is allowed to commit now
2015-02-12 12:04:11,555 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1736216983_0031_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator/_temporary/0/task_local1736216983_0031_r_000000
2015-02-12 12:04:11,556 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:11,556 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1736216983_0031_r_000000_0' done.
2015-02-12 12:04:11,556 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1736216983_0031_r_000000_0
2015-02-12 12:04:11,556 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:11,574 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator/_temporary
2015-02-12 12:04:11,603 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:11,604 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator
2015-02-12 12:04:11,622 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:11,622 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:11,638 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:11,640 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:11,660 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:11,664 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...] starting
2015-02-12 12:04:11,665 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:11,665 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...]  sink: Hfs["TextDelimited[['char', 'count']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway"]
2015-02-12 12:04:11,665 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...]  parallel execution is enabled: false
2015-02-12 12:04:11,665 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...]  starting jobs: 5
2015-02-12 12:04:11,665 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...]  allocating threads: 1
2015-02-12 12:04:11,665 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (1/5)
2015-02-12 12:04:11,666 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:11,667 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:11,687 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:11,706 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:11,720 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local1903952708_0032
2015-02-12 12:04:11,772 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:11,773 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:11,773 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:11,773 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1903952708_0032
2015-02-12 12:04:11,773 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-12 12:04:11,775 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:11,775 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1903952708_0032_m_000000_0
2015-02-12 12:04:11,776 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:11,776 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:11,777 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:04:11,777 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@43453754
2015-02-12 12:04:11,777 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:11,796 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:11,796 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:11,796 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:11,796 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:11,797 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:11,797 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:11,797 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:11,797 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:11,802 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:11,802 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(source)[by:[{1}:'char']]
2015-02-12 12:04:11,802 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:11,803 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:11,803 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 159, total: 573, max: 672
2015-02-12 12:04:11,804 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:11,804 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 159, total: 573, max: 672
2015-02-12 12:04:11,804 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-12 12:04:11,804 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 159, total: 573, max: 672
2015-02-12 12:04:11,805 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-12 12:04:11,805 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 159, total: 573, max: 672
2015-02-12 12:04:11,806 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:11,806 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:11,806 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:11,806 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-12 12:04:11,806 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-12 12:04:11,808 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:11,809 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1903952708_0032_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:11,810 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:04:11,811 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1903952708_0032_m_000000_0' done.
2015-02-12 12:04:11,811 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1903952708_0032_m_000000_0
2015-02-12 12:04:11,811 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:11,811 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:11,811 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1903952708_0032_r_000000_0
2015-02-12 12:04:11,812 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:11,813 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:11,813 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@24943658
2015-02-12 12:04:11,813 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:11,814 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1903952708_0032_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:11,815 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#25 about to shuffle output of map attempt_local1903952708_0032_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-12 12:04:11,815 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local1903952708_0032_m_000000_0
2015-02-12 12:04:11,815 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-12 12:04:11,816 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:11,817 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:11,817 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:11,818 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:11,818 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-12 12:04:11,819 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:11,819 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 123 bytes from disk
2015-02-12 12:04:11,819 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:11,819 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:11,820 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-12 12:04:11,820 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:11,820 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:11,821 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:11,825 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(source)[by:[{1}:'char']]
2015-02-12 12:04:11,825 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][5591123506/source/]
2015-02-12 12:04:11,833 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1903952708_0032_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:11,834 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:11,834 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local1903952708_0032_r_000000_0 is allowed to commit now
2015-02-12 12:04:11,835 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1903952708_0032_r_000000_0' to file:/tmp/hadoop-soberoi/5591123506_source_7500BA352D8C4845B36E47E96C0B0809/_temporary/0/task_local1903952708_0032_r_000000
2015-02-12 12:04:11,836 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:11,836 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1903952708_0032_r_000000_0' done.
2015-02-12 12:04:11,836 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1903952708_0032_r_000000_0
2015-02-12 12:04:11,836 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:11,856 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (3/5)
2015-02-12 12:04:11,857 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:11,858 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:11,895 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:11,911 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:11,919 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:11,920 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:11,923 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local596787385_0033
2015-02-12 12:04:11,972 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:11,972 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local596787385_0033
2015-02-12 12:04:11,972 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-12 12:04:11,973 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:11,973 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:11,975 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:11,975 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local596787385_0033_m_000000_0
2015-02-12 12:04:11,976 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:11,976 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:11,976 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-soberoi/5591123506_source_7500BA352D8C4845B36E47E96C0B0809/part-00000
2015-02-12 12:04:11,977 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5911e373
2015-02-12 12:04:11,977 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:11,995 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:11,995 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:11,995 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:11,995 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:11,995 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:11,996 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:11,996 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:11,996 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:12,000 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][5591123506/source/]
2015-02-12 12:04:12,000 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-lhs)[by:[{1}:'char']]
2015-02-12 12:04:12,000 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:12,001 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:12,001 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 161, total: 574, max: 672
2015-02-12 12:04:12,002 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:12,002 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 161, total: 574, max: 672
2015-02-12 12:04:12,003 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:12,003 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:12,003 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:12,003 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 55; bufvoid = 104857600
2015-02-12 12:04:12,003 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-12 12:04:12,004 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:12,006 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local596787385_0033_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:12,007 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-soberoi/5591123506_source_7500BA352D8C4845B36E47E96C0B0809/part-00000:0+160
2015-02-12 12:04:12,007 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local596787385_0033_m_000000_0' done.
2015-02-12 12:04:12,007 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local596787385_0033_m_000000_0
2015-02-12 12:04:12,008 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:12,008 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:12,008 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local596787385_0033_r_000000_0
2015-02-12 12:04:12,009 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:12,009 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:12,009 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@1b788dd2
2015-02-12 12:04:12,010 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:12,010 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local596787385_0033_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:12,011 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#26 about to shuffle output of map attempt_local596787385_0033_m_000000_0 decomp: 67 len: 71 to MEMORY
2015-02-12 12:04:12,012 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 67 bytes from map-output for attempt_local596787385_0033_m_000000_0
2015-02-12 12:04:12,012 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 67, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->67
2015-02-12 12:04:12,012 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:12,013 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:12,013 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:12,014 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:12,014 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-12 12:04:12,015 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 67 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:12,015 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 71 bytes from disk
2015-02-12 12:04:12,016 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:12,016 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:12,016 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-12 12:04:12,016 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:12,017 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:12,017 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:12,021 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count-lhs)[by:[{1}:'char']]
2015-02-12 12:04:12,021 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][9079103016/count-lhs/]
2015-02-12 12:04:12,030 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local596787385_0033_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:12,031 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:12,031 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local596787385_0033_r_000000_0 is allowed to commit now
2015-02-12 12:04:12,031 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local596787385_0033_r_000000_0' to file:/tmp/hadoop-soberoi/9079103016_count_lhs_000FA2B39D9F404CA5B406B076CEB9D0/_temporary/0/task_local596787385_0033_r_000000
2015-02-12 12:04:12,032 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:12,032 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local596787385_0033_r_000000_0' done.
2015-02-12 12:04:12,032 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local596787385_0033_r_000000_0
2015-02-12 12:04:12,033 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:12,052 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (4/5)
2015-02-12 12:04:12,052 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:12,053 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:12,082 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:12,098 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:12,107 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:12,107 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:12,112 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local717631152_0034
2015-02-12 12:04:12,158 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:12,159 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:12,159 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:12,159 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local717631152_0034
2015-02-12 12:04:12,159 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-12 12:04:12,160 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:12,160 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local717631152_0034_m_000000_0
2015-02-12 12:04:12,161 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:12,162 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:12,163 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-soberoi/5591123506_source_7500BA352D8C4845B36E47E96C0B0809/part-00000
2015-02-12 12:04:12,163 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1408e45b
2015-02-12 12:04:12,164 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:12,180 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:12,181 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:12,181 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:12,181 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:12,181 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:12,181 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:12,182 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:12,182 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:12,185 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][5591123506/source/]
2015-02-12 12:04:12,185 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-rhs)[by:[{1}:'char']]
2015-02-12 12:04:12,185 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:12,186 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 12:04:12,186 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 161, total: 574, max: 672
2015-02-12 12:04:12,186 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 12:04:12,187 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 161, total: 574, max: 672
2015-02-12 12:04:12,187 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:12,188 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:12,188 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:12,188 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 55; bufvoid = 104857600
2015-02-12 12:04:12,188 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-12 12:04:12,189 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:12,190 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local717631152_0034_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:12,191 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-soberoi/5591123506_source_7500BA352D8C4845B36E47E96C0B0809/part-00000:0+160
2015-02-12 12:04:12,191 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local717631152_0034_m_000000_0' done.
2015-02-12 12:04:12,192 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local717631152_0034_m_000000_0
2015-02-12 12:04:12,192 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:12,192 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:12,192 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local717631152_0034_r_000000_0
2015-02-12 12:04:12,193 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:12,193 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:12,193 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@604dc0bb
2015-02-12 12:04:12,194 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:12,194 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local717631152_0034_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:12,195 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#27 about to shuffle output of map attempt_local717631152_0034_m_000000_0 decomp: 67 len: 71 to MEMORY
2015-02-12 12:04:12,196 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 67 bytes from map-output for attempt_local717631152_0034_m_000000_0
2015-02-12 12:04:12,196 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 67, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->67
2015-02-12 12:04:12,196 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:12,197 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:12,197 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:12,198 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:12,198 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-12 12:04:12,198 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 67 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:12,199 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 71 bytes from disk
2015-02-12 12:04:12,199 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:12,199 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:12,199 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-12 12:04:12,200 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:12,200 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:12,200 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:12,204 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count-rhs)[by:[{1}:'char']]
2015-02-12 12:04:12,204 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][1577092611/count-rhs/]
2015-02-12 12:04:12,213 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local717631152_0034_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:12,214 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:12,214 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local717631152_0034_r_000000_0 is allowed to commit now
2015-02-12 12:04:12,215 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local717631152_0034_r_000000_0' to file:/tmp/hadoop-soberoi/1577092611_count_rhs_7C5C499AAA294973B5B0543AD89BDF3B/_temporary/0/task_local717631152_0034_r_000000
2015-02-12 12:04:12,215 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:12,216 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local717631152_0034_r_000000_0' done.
2015-02-12 12:04:12,216 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local717631152_0034_r_000000_0
2015-02-12 12:04:12,216 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:12,234 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (2/5)
2015-02-12 12:04:12,234 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:12,235 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:12,280 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:12,293 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:12,313 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:2
2015-02-12 12:04:12,325 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:12,325 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:12,330 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local600792159_0035
2015-02-12 12:04:12,381 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:12,381 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:12,382 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local600792159_0035
2015-02-12 12:04:12,382 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:12,382 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-12 12:04:12,384 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:12,384 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local600792159_0035_m_000000_0
2015-02-12 12:04:12,385 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:12,385 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:12,385 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-soberoi/1577092611_count_rhs_7C5C499AAA294973B5B0543AD89BDF3B/part-00000
2015-02-12 12:04:12,386 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@499fa9b0
2015-02-12 12:04:12,386 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:12,402 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:12,403 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:12,403 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:12,403 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:12,403 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:12,405 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:12,405 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:12,405 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:12,408 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][1577092611/count-rhs/]
2015-02-12 12:04:12,409 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-12 12:04:12,410 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:12,410 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:12,410 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:12,410 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 65; bufvoid = 104857600
2015-02-12 12:04:12,410 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-12 12:04:12,412 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:12,413 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local600792159_0035_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:12,414 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-soberoi/1577092611_count_rhs_7C5C499AAA294973B5B0543AD89BDF3B/part-00000:0+160
2015-02-12 12:04:12,414 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local600792159_0035_m_000000_0' done.
2015-02-12 12:04:12,414 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local600792159_0035_m_000000_0
2015-02-12 12:04:12,414 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local600792159_0035_m_000001_0
2015-02-12 12:04:12,415 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:12,415 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:12,416 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-soberoi/5591123506_source_7500BA352D8C4845B36E47E96C0B0809/part-00000
2015-02-12 12:04:12,416 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@467924cb
2015-02-12 12:04:12,417 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:12,430 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:12,430 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:12,430 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:12,430 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:12,430 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:12,431 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:12,431 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:12,431 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:12,435 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][5591123506/source/]
2015-02-12 12:04:12,435 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-12 12:04:12,437 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:12,437 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:12,437 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:12,437 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 65; bufvoid = 104857600
2015-02-12 12:04:12,437 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-12 12:04:12,438 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:12,439 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local600792159_0035_m_000001_0 is done. And is in the process of committing
2015-02-12 12:04:12,440 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-soberoi/5591123506_source_7500BA352D8C4845B36E47E96C0B0809/part-00000:0+160
2015-02-12 12:04:12,440 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local600792159_0035_m_000001_0' done.
2015-02-12 12:04:12,440 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local600792159_0035_m_000001_0
2015-02-12 12:04:12,440 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:12,441 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:12,441 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local600792159_0035_r_000000_0
2015-02-12 12:04:12,442 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:12,442 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:12,442 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@4f643703
2015-02-12 12:04:12,443 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:12,443 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local600792159_0035_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:12,445 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#28 about to shuffle output of map attempt_local600792159_0035_m_000001_0 decomp: 77 len: 81 to MEMORY
2015-02-12 12:04:12,445 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 77 bytes from map-output for attempt_local600792159_0035_m_000001_0
2015-02-12 12:04:12,446 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 77, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->77
2015-02-12 12:04:12,446 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#28 about to shuffle output of map attempt_local600792159_0035_m_000000_0 decomp: 77 len: 81 to MEMORY
2015-02-12 12:04:12,447 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 77 bytes from map-output for attempt_local600792159_0035_m_000000_0
2015-02-12 12:04:12,447 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 77, inMemoryMapOutputs.size() -> 2, commitMemory -> 77, usedMemory ->154
2015-02-12 12:04:12,447 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:12,448 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:04:12,448 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:12,449 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 2 sorted segments
2015-02-12 12:04:12,449 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 2 segments left of total size: 134 bytes
2015-02-12 12:04:12,450 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 2 segments, 154 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:12,451 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 156 bytes from disk
2015-02-12 12:04:12,451 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:12,451 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:12,451 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 142 bytes
2015-02-12 12:04:12,452 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:04:12,452 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:12,452 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:12,457 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-12 12:04:12,457 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'num', 'char2', 'count']]"][1522260838/source_count-rhs/]
2015-02-12 12:04:12,468 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(105)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-12 12:04:12,468 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(110)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-12 12:04:12,472 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local600792159_0035_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:12,473 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:04:12,473 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local600792159_0035_r_000000_0 is allowed to commit now
2015-02-12 12:04:12,474 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local600792159_0035_r_000000_0' to file:/tmp/hadoop-soberoi/1522260838_source_count_rhs_7BC0AC8384FF4A1492BECEB460EBD4AF/_temporary/0/task_local600792159_0035_r_000000
2015-02-12 12:04:12,474 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:12,475 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local600792159_0035_r_000000_0' done.
2015-02-12 12:04:12,475 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local600792159_0035_r_000000_0
2015-02-12 12:04:12,475 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:12,494 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (5/5) ...m/samesourcemergethreeway
2015-02-12 12:04:12,495 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:12,496 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:12,535 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:12,548 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:12,572 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:2
2015-02-12 12:04:12,582 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:12,583 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:12,588 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local1459280829_0036
2015-02-12 12:04:12,664 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:12,664 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:12,664 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:12,665 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1459280829_0036
2015-02-12 12:04:12,665 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-12 12:04:12,666 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:12,666 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1459280829_0036_m_000000_0
2015-02-12 12:04:12,667 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:12,667 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:12,668 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-soberoi/1522260838_source_count_rhs_7BC0AC8384FF4A1492BECEB460EBD4AF/part-00000
2015-02-12 12:04:12,668 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6045a3ce
2015-02-12 12:04:12,669 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:12,685 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:12,685 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:12,685 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:12,685 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:12,685 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:12,686 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:12,686 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:12,686 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:12,692 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'num', 'char2', 'count']]"][1522260838/source_count-rhs/]
2015-02-12 12:04:12,692 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2015-02-12 12:04:12,693 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:12,694 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:12,694 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:12,694 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 105; bufvoid = 104857600
2015-02-12 12:04:12,694 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-12 12:04:12,695 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:12,696 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1459280829_0036_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:12,697 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-soberoi/1522260838_source_count_rhs_7BC0AC8384FF4A1492BECEB460EBD4AF/part-00000:0+200
2015-02-12 12:04:12,697 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1459280829_0036_m_000000_0' done.
2015-02-12 12:04:12,698 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1459280829_0036_m_000000_0
2015-02-12 12:04:12,698 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1459280829_0036_m_000001_0
2015-02-12 12:04:12,699 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:12,699 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:12,700 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-soberoi/9079103016_count_lhs_000FA2B39D9F404CA5B406B076CEB9D0/part-00000
2015-02-12 12:04:12,700 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@66d7e31d
2015-02-12 12:04:12,701 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:12,714 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:12,715 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:12,715 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:12,715 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:12,715 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:12,715 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:12,716 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:12,716 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:12,721 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][9079103016/count-lhs/]
2015-02-12 12:04:12,722 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2015-02-12 12:04:12,723 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:12,723 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:12,724 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:12,724 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 65; bufvoid = 104857600
2015-02-12 12:04:12,724 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-12 12:04:12,725 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:12,726 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1459280829_0036_m_000001_0 is done. And is in the process of committing
2015-02-12 12:04:12,728 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-soberoi/9079103016_count_lhs_000FA2B39D9F404CA5B406B076CEB9D0/part-00000:0+160
2015-02-12 12:04:12,728 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1459280829_0036_m_000001_0' done.
2015-02-12 12:04:12,728 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1459280829_0036_m_000001_0
2015-02-12 12:04:12,728 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:12,728 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:12,729 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1459280829_0036_r_000000_0
2015-02-12 12:04:12,730 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:12,730 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:12,730 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@11c3765c
2015-02-12 12:04:12,731 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:12,732 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1459280829_0036_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:12,733 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#29 about to shuffle output of map attempt_local1459280829_0036_m_000001_0 decomp: 77 len: 81 to MEMORY
2015-02-12 12:04:12,733 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 77 bytes from map-output for attempt_local1459280829_0036_m_000001_0
2015-02-12 12:04:12,733 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 77, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->77
2015-02-12 12:04:12,735 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#29 about to shuffle output of map attempt_local1459280829_0036_m_000000_0 decomp: 117 len: 121 to MEMORY
2015-02-12 12:04:12,735 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 117 bytes from map-output for attempt_local1459280829_0036_m_000000_0
2015-02-12 12:04:12,735 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 117, inMemoryMapOutputs.size() -> 2, commitMemory -> 77, usedMemory ->194
2015-02-12 12:04:12,735 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:12,736 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:04:12,736 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:12,737 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 2 sorted segments
2015-02-12 12:04:12,737 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 2 segments left of total size: 174 bytes
2015-02-12 12:04:12,738 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 2 segments, 194 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:12,739 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 196 bytes from disk
2015-02-12 12:04:12,739 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:12,739 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:12,739 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 182 bytes
2015-02-12 12:04:12,740 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:04:12,740 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:12,740 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:12,745 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2015-02-12 12:04:12,745 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway"]
2015-02-12 12:04:12,753 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(105)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-12 12:04:12,753 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(110)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-12 12:04:12,755 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1459280829_0036_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:12,756 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:04:12,756 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local1459280829_0036_r_000000_0 is allowed to commit now
2015-02-12 12:04:12,757 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1459280829_0036_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway/_temporary/0/task_local1459280829_0036_r_000000
2015-02-12 12:04:12,758 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:12,758 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1459280829_0036_r_000000_0' done.
2015-02-12 12:04:12,758 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1459280829_0036_r_000000_0
2015-02-12 12:04:12,758 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:12,784 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway/_temporary
2015-02-12 12:04:12,819 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:12,853 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:12,854 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway
2015-02-12 12:04:12,875 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:12,876 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:12,882 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:12,884 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:12,884 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:12,891 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs] starting
2015-02-12 12:04:12,891 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt"]
2015-02-12 12:04:12,891 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:12,891 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs]  sink: Hfs["TextLine[['item']->['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic"]
2015-02-12 12:04:12,891 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs]  parallel execution is enabled: false
2015-02-12 12:04:12,892 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs]  starting jobs: 1
2015-02-12 12:04:12,892 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs]  allocating threads: 1
2015-02-12 12:04:12,892 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lhs+rhs] starting step: (1/1) ...quemerge-nondeterministic
2015-02-12 12:04:12,893 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:12,894 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:12,921 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:12,922 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:12,941 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:2
2015-02-12 12:04:12,956 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local325901911_0037
2015-02-12 12:04:13,003 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:13,003 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:13,004 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:13,004 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lhs+rhs] submitted hadoop job: job_local325901911_0037
2015-02-12 12:04:13,004 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lhs+rhs] tracking url: http://localhost:8080/
2015-02-12 12:04:13,005 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:13,005 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local325901911_0037_m_000000_0
2015-02-12 12:04:13,006 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:13,006 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:13,007 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:04:13,007 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@12e6d8aa
2015-02-12 12:04:13,008 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:13,019 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:13,019 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:13,019 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:13,019 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:13,019 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:13,020 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:13,020 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:13,020 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:13,023 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:13,024 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2015-02-12 12:04:13,027 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:13,027 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:13,027 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:13,027 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-12 12:04:13,027 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-12 12:04:13,028 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:13,030 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local325901911_0037_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:13,031 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:04:13,031 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local325901911_0037_m_000000_0' done.
2015-02-12 12:04:13,031 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local325901911_0037_m_000000_0
2015-02-12 12:04:13,031 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local325901911_0037_m_000001_0
2015-02-12 12:04:13,032 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:13,032 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:13,033 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt
2015-02-12 12:04:13,033 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2220ccac
2015-02-12 12:04:13,034 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:13,051 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:13,051 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:13,051 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:13,051 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:13,051 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:13,052 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:13,052 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:13,052 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:13,056 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt"]
2015-02-12 12:04:13,056 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2015-02-12 12:04:13,058 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:13,059 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:13,059 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:13,059 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-12 12:04:13,059 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-12 12:04:13,060 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:13,061 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local325901911_0037_m_000001_0 is done. And is in the process of committing
2015-02-12 12:04:13,063 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/rhs.txt:0+51
2015-02-12 12:04:13,063 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local325901911_0037_m_000001_0' done.
2015-02-12 12:04:13,063 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local325901911_0037_m_000001_0
2015-02-12 12:04:13,063 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:13,064 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:13,064 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local325901911_0037_r_000000_0
2015-02-12 12:04:13,065 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:13,066 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:13,066 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@564e8791
2015-02-12 12:04:13,066 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:13,067 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local325901911_0037_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:13,068 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#30 about to shuffle output of map attempt_local325901911_0037_m_000000_0 decomp: 87 len: 91 to MEMORY
2015-02-12 12:04:13,069 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local325901911_0037_m_000000_0
2015-02-12 12:04:13,069 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2015-02-12 12:04:13,070 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#30 about to shuffle output of map attempt_local325901911_0037_m_000001_0 decomp: 87 len: 91 to MEMORY
2015-02-12 12:04:13,070 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local325901911_0037_m_000001_0
2015-02-12 12:04:13,070 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 2, commitMemory -> 87, usedMemory ->174
2015-02-12 12:04:13,071 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:13,072 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:04:13,072 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:13,073 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 2 sorted segments
2015-02-12 12:04:13,073 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 2 segments left of total size: 156 bytes
2015-02-12 12:04:13,074 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 2 segments, 174 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:13,074 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 176 bytes from disk
2015-02-12 12:04:13,075 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:13,075 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:13,075 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 163 bytes
2015-02-12 12:04:13,075 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:04:13,076 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:13,076 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:13,080 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2015-02-12 12:04:13,080 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['item']->['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic"]
2015-02-12 12:04:13,090 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local325901911_0037_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:13,091 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 12:04:13,092 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local325901911_0037_r_000000_0 is allowed to commit now
2015-02-12 12:04:13,092 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local325901911_0037_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic/_temporary/0/task_local325901911_0037_r_000000
2015-02-12 12:04:13,093 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:13,093 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local325901911_0037_r_000000_0' done.
2015-02-12 12:04:13,093 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local325901911_0037_r_000000_0
2015-02-12 12:04:13,093 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:13,116 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic/_temporary
2015-02-12 12:04:13,152 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:13,153 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic
2015-02-12 12:04:13,171 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:13,172 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:13,177 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:13,179 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:13,179 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:13,182 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-12 12:04:13,182 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:13,182 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  sink: Hfs["TextLine[['item']->['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique"]
2015-02-12 12:04:13,182 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  parallel execution is enabled: false
2015-02-12 12:04:13,182 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  starting jobs: 1
2015-02-12 12:04:13,182 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  allocating threads: 1
2015-02-12 12:04:13,183 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...blyhelpersplatform/unique
2015-02-12 12:04:13,183 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:13,184 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:13,206 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:13,222 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:13,235 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local911178289_0038
2015-02-12 12:04:13,295 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:13,298 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:13,298 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:13,298 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local911178289_0038
2015-02-12 12:04:13,299 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-12 12:04:13,300 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:13,300 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local911178289_0038_m_000000_0
2015-02-12 12:04:13,301 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:13,302 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:13,303 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:04:13,303 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2ce683c
2015-02-12 12:04:13,304 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:13,320 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:13,321 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:13,321 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:13,321 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:13,321 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:13,322 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:13,322 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:13,322 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:13,325 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:13,325 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(shape)[by:[{1}:'num']]
2015-02-12 12:04:13,328 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:13,328 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:13,328 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:13,328 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-12 12:04:13,328 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-12 12:04:13,330 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:13,331 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local911178289_0038_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:13,332 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:04:13,332 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local911178289_0038_m_000000_0' done.
2015-02-12 12:04:13,332 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local911178289_0038_m_000000_0
2015-02-12 12:04:13,333 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:13,333 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:13,333 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local911178289_0038_r_000000_0
2015-02-12 12:04:13,335 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:13,336 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:13,336 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@1d7b0a74
2015-02-12 12:04:13,336 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:13,337 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local911178289_0038_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:13,338 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#31 about to shuffle output of map attempt_local911178289_0038_m_000000_0 decomp: 87 len: 91 to MEMORY
2015-02-12 12:04:13,338 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local911178289_0038_m_000000_0
2015-02-12 12:04:13,338 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2015-02-12 12:04:13,339 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:13,339 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:13,339 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:13,340 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:13,341 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-12 12:04:13,341 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 87 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:13,342 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 91 bytes from disk
2015-02-12 12:04:13,342 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:13,342 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:13,342 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-12 12:04:13,342 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:13,343 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:13,343 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:13,346 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(shape)[by:[{1}:'num']]
2015-02-12 12:04:13,346 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['item']->['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique"]
2015-02-12 12:04:13,354 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local911178289_0038_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:13,355 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:13,356 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local911178289_0038_r_000000_0 is allowed to commit now
2015-02-12 12:04:13,356 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local911178289_0038_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique/_temporary/0/task_local911178289_0038_r_000000
2015-02-12 12:04:13,357 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:13,357 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local911178289_0038_r_000000_0' done.
2015-02-12 12:04:13,357 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local911178289_0038_r_000000_0
2015-02-12 12:04:13,357 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:13,379 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique/_temporary
2015-02-12 12:04:13,412 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:13,413 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/unique
2015-02-12 12:04:13,436 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 12:04:13,436 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 12:04:13,441 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 12:04:13,442 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 12:04:13,442 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 12:04:13,446 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting
2015-02-12 12:04:13,447 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  source: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:13,447 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  sink: Hfs["TextDelimited[['notnull', 'null']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull"]
2015-02-12 12:04:13,447 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  parallel execution is enabled: false
2015-02-12 12:04:13,447 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  starting jobs: 1
2015-02-12 12:04:13,447 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  allocating threads: 1
2015-02-12 12:04:13,447 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...platform/countnullnotnull
2015-02-12 12:04:13,448 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:13,449 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 12:04:13,473 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:13,496 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-02-12 12:04:13,513 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_local502043800_0039
2015-02-12 12:04:13,576 INFO  mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://localhost:8080/
2015-02-12 12:04:13,576 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 12:04:13,576 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 12:04:13,576 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local502043800_0039
2015-02-12 12:04:13,577 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-12 12:04:13,577 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 12:04:13,577 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local502043800_0039_m_000000_0
2015-02-12 12:04:13,578 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:13,578 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:13,579 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt
2015-02-12 12:04:13,579 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@29a8186c
2015-02-12 12:04:13,580 INFO  mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 12:04:13,597 INFO  mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 12:04:13,597 INFO  mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 12:04:13,597 INFO  mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 12:04:13,597 INFO  mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 12:04:13,598 INFO  mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 12:04:13,598 INFO  mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 12:04:13,598 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 12:04:13,599 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 12:04:13,604 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt"]
2015-02-12 12:04:13,604 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{?}:NONE]]
2015-02-12 12:04:13,604 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 12:04:13,609 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-12 12:04:13,609 INFO  mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 12:04:13,609 INFO  mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 12:04:13,609 INFO  mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 6; bufvoid = 104857600
2015-02-12 12:04:13,609 INFO  mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214396(104857584); length = 1/6553600
2015-02-12 12:04:13,611 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 12:04:13,612 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local502043800_0039_m_000000_0 is done. And is in the process of committing
2015-02-12 12:04:13,613 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/classes/test/data/lhs.txt:0+52
2015-02-12 12:04:13,613 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local502043800_0039_m_000000_0' done.
2015-02-12 12:04:13,613 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local502043800_0039_m_000000_0
2015-02-12 12:04:13,614 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 12:04:13,614 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 12:04:13,614 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local502043800_0039_r_000000_0
2015-02-12 12:04:13,615 INFO  util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 12:04:13,615 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : null
2015-02-12 12:04:13,615 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@11a59f5c
2015-02-12 12:04:13,616 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(196)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 12:04:13,616 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local502043800_0039_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 12:04:13,617 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#32 about to shuffle output of map attempt_local502043800_0039_m_000000_0 decomp: 10 len: 14 to MEMORY
2015-02-12 12:04:13,617 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 10 bytes from map-output for attempt_local502043800_0039_m_000000_0
2015-02-12 12:04:13,618 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(314)) - closeInMemoryFile -> map-output of size: 10, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->10
2015-02-12 12:04:13,618 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 12:04:13,618 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:13,619 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(674)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 12:04:13,620 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:13,620 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 7 bytes
2015-02-12 12:04:13,621 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(751)) - Merged 1 segments, 10 bytes to disk to satisfy reduce memory limit
2015-02-12 12:04:13,621 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(781)) - Merging 1 files, 14 bytes from disk
2015-02-12 12:04:13,621 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(796)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 12:04:13,621 INFO  mapred.Merger (Merger.java:merge(597)) - Merging 1 sorted segments
2015-02-12 12:04:13,621 INFO  mapred.Merger (Merger.java:merge(696)) - Down to the last merge-pass, with 1 segments left of total size: 7 bytes
2015-02-12 12:04:13,622 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:13,622 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 12:04:13,622 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 12:04:13,628 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{?}:NONE]]
2015-02-12 12:04:13,628 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['notnull', 'null']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull"]
2015-02-12 12:04:13,636 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local502043800_0039_r_000000_0 is done. And is in the process of committing
2015-02-12 12:04:13,637 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 12:04:13,637 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local502043800_0039_r_000000_0 is allowed to commit now
2015-02-12 12:04:13,638 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local502043800_0039_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull/_temporary/0/task_local502043800_0039_r_000000
2015-02-12 12:04:13,639 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 12:04:13,639 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local502043800_0039_r_000000_0' done.
2015-02-12 12:04:13,639 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local502043800_0039_r_000000_0
2015-02-12 12:04:13,639 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 12:04:13,653 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull/_temporary
2015-02-12 12:04:13,685 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:13,720 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 12:04:13,721 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/horton/build/test/output//hadoop2-mr1/assemblyhelpersplatform/countnullnotnull